d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1b108b19ed
freebsd-dev/sys/powerpc/booke/pmap.c

4384 lines
110 KiB
C
Raw Normal View History

Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*-
sys/powerpc: further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 2-Clause license, however the tool I was using misidentified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts.
2017-11-27 15:09:59 +00:00
* SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
* Copyright (C) 2007-2009 Semihalf, Rafal Jaworowski <raj@semihalf.com>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* Copyright (C) 2006 Semihalf, Marian Balakowicz <m8@semihalf.com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN
* NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
* TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
* Some hw specific parts of this pmap were derived or influenced
* by NetBSD's ibm4xx pmap module. More generic code is shared with
* a few other pmap modules from the FreeBSD tree.
*/
/*
* VM layout notes:
*
* Kernel and user threads run within one common virtual address space
* defined by AS=0.
*
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
* 32-bit pmap:
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* Virtual address space layout:
* -----------------------------
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
* 0x0000_0000 - 0x7fff_ffff : user process
* 0x8000_0000 - 0xbfff_ffff : pmap_mapdev()-ed area (PCI/PCIE etc.)
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* 0xc000_0000 - 0xc0ff_ffff : kernel reserved
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
* 0xc000_0000 - data_end : kernel code+data, env, metadata etc.
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
* 0xc100_0000 - 0xffff_ffff : KVA
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* 0xc100_0000 - 0xc100_3fff : reserved for page zero/copy
* 0xc100_4000 - 0xc200_3fff : reserved for ptbl bufs
* 0xc200_4000 - 0xc200_8fff : guard page + kstack0
* 0xc200_9000 - 0xfeef_ffff : actual free KVA space
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
*
* 64-bit pmap:
* Virtual address space layout:
* -----------------------------
* 0x0000_0000_0000_0000 - 0xbfff_ffff_ffff_ffff : user process
* 0x0000_0000_0000_0000 - 0x8fff_ffff_ffff_ffff : text, data, heap, maps, libraries
* 0x9000_0000_0000_0000 - 0xafff_ffff_ffff_ffff : mmio region
* 0xb000_0000_0000_0000 - 0xbfff_ffff_ffff_ffff : stack
* 0xc000_0000_0000_0000 - 0xcfff_ffff_ffff_ffff : kernel reserved
* 0xc000_0000_0000_0000 - endkernel-1 : kernel code & data
* endkernel - msgbufp-1 : flat device tree
* msgbufp - ptbl_bufs-1 : message buffer
* ptbl_bufs - kernel_pdir-1 : kernel page tables
* kernel_pdir - kernel_pp2d-1 : kernel page directory
* kernel_pp2d - . : kernel pointers to page directory
* pmap_zero_copy_min - crashdumpmap-1 : reserved for page zero/copy
* crashdumpmap - ptbl_buf_pool_vabase-1 : reserved for ptbl bufs
* ptbl_buf_pool_vabase - virtual_avail-1 : user page directories and page tables
* virtual_avail - 0xcfff_ffff_ffff_ffff : actual free KVA space
* 0xd000_0000_0000_0000 - 0xdfff_ffff_ffff_ffff : coprocessor region
* 0xe000_0000_0000_0000 - 0xefff_ffff_ffff_ffff : mmio region
* 0xf000_0000_0000_0000 - 0xffff_ffff_ffff_ffff : direct map
* 0xf000_0000_0000_0000 - +Maxmem : physmem map
* - 0xffff_ffff_ffff_ffff : device direct map
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
Fix KSTACK_PAGES issue when the default value was changed in KERNCONF If KSTACK_PAGES was changed to anything alse than the default, the value from param.h was taken instead in some places and the value from KENRCONF in some others. This resulted in inconsistency which caused corruption in SMP envorinment. Ensure all places where KSTACK_PAGES are used the opt_kstack_pages.h is included. The file opt_kstack_pages.h could not be included in param.h because was breaking the toolchain compilation. Reviewed by: kib Obtained from: Semihalf Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D3094
2015-07-16 10:46:52 +00:00
#include "opt_kstack_pages.h"
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/param.h>
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
#include <sys/conf.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/malloc.h>
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
#include <sys/ktr.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/proc.h>
#include <sys/user.h>
#include <sys/queue.h>
#include <sys/systm.h>
#include <sys/kernel.h>
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
#include <sys/kerneldump.h>
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
#include <sys/linker.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/msgbuf.h>
#include <sys/lock.h>
#include <sys/mutex.h>
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
#include <sys/rwlock.h>
MFC
2011-05-17 22:03:01 +00:00
#include <sys/sched.h>
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
#include <sys/smp.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/vmmeter.h>
#include <vm/vm.h>
#include <vm/vm_page.h>
#include <vm/vm_kern.h>
#include <vm/vm_pageout.h>
#include <vm/vm_extern.h>
#include <vm/vm_object.h>
#include <vm/vm_param.h>
#include <vm/vm_map.h>
#include <vm/vm_pager.h>
#include <vm/uma.h>
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#include <machine/_inttypes.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <machine/cpu.h>
#include <machine/pcb.h>
Factor out platform dependent things unrelated to device drivers into a new platform module. These are probed in early boot, and have the responsibility of determining the layout of physical memory, determining the CPU timebase frequency, and handling the zoo of SMP mechanisms found on PowerPC. Reviewed by: marcel, raj Book-E parts by: raj
2009-05-14 00:34:26 +00:00
#include <machine/platform.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <machine/tlb.h>
#include <machine/spr.h>
#include <machine/md_var.h>
#include <machine/mmuvar.h>
#include <machine/pmap.h>
#include <machine/pte.h>
#include "mmu_if.h"
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
#define SPARSE_MAPDEV
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#ifdef DEBUG
#define debugf(fmt, args...) printf(fmt, ##args)
#else
#define debugf(fmt, args...)
#endif
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
#define PRI0ptrX "016lx"
#else
#define PRI0ptrX "08x"
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#define TODO panic("%s: not implemented", __func__);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
extern unsigned char _etext[];
extern unsigned char _end[];
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
extern uint32_t *bootinfo;
vm_paddr_t kernload;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
vm_offset_t kernstart;
vm_size_t kernsize;
/* Message buffer and tables. */
static vm_offset_t data_start;
static vm_size_t data_end;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Factor out platform dependent things unrelated to device drivers into a new platform module. These are probed in early boot, and have the responsibility of determining the layout of physical memory, determining the CPU timebase frequency, and handling the zoo of SMP mechanisms found on PowerPC. Reviewed by: marcel, raj Book-E parts by: raj
2009-05-14 00:34:26 +00:00
/* Phys/avail memory regions. */
static struct mem_region *availmem_regions;
static int availmem_regions_sz;
static struct mem_region *physmem_regions;
static int physmem_regions_sz;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Reserved KVA space and mutex for mmu_booke_zero_page. */
static vm_offset_t zero_page_va;
static struct mtx zero_page_mutex;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static struct mtx tlbivax_mutex;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Reserved KVA space and mutex for mmu_booke_copy_page. */
static vm_offset_t copy_page_src_va;
static vm_offset_t copy_page_dst_va;
static struct mtx copy_page_mutex;
/**************************************************************************/
/* PMAP */
/**************************************************************************/
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
static int mmu_booke_enter_locked(mmu_t, pmap_t, vm_offset_t, vm_page_t,
vm_prot_t, u_int flags, int8_t psind);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
unsigned int kptbl_min; /* Index of the first kernel ptbl. */
unsigned int kernel_ptbls; /* Number of KVA ptbls. */
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
unsigned int kernel_pdirs;
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* If user pmap is processed with mmu_booke_remove and the resident count
* drops to 0, there are no more pages to remove, so we need not continue.
*/
#define PMAP_REMOVE_DONE(pmap) \
((pmap) != kernel_pmap && (pmap)->pm_stats.resident_count == 0)
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#if defined(COMPAT_FREEBSD32) || !defined(__powerpc64__)
Missed local diff.
2015-03-01 21:47:38 +00:00
extern int elf32_nxstack;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/**************************************************************************/
/* TLB and TID handling */
/**************************************************************************/
/* Translation ID busy table */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static volatile pmap_t tidbusy[MAXCPU][TID_MAX + 1];
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* TLB0 capabilities (entry, way numbers etc.). These can vary between e500
* core revisions and should be read from h/w registers during early config.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
uint32_t tlb0_entries;
uint32_t tlb0_ways;
uint32_t tlb0_entries_per_way;
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
uint32_t tlb1_entries;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
#define TLB0_ENTRIES (tlb0_entries)
#define TLB0_WAYS (tlb0_ways)
#define TLB0_ENTRIES_PER_WAY (tlb0_entries_per_way)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
#define TLB1_ENTRIES (tlb1_entries)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
VM_MAXUSER_ADDRESS is highest page start, not highest address. In case a single page mapping is requested first, which might overlap the user address space, fix the device map block to the next page.
2016-04-10 15:50:45 +00:00
static vm_offset_t tlb1_map_base = VM_MAXUSER_ADDRESS + PAGE_SIZE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static tlbtid_t tid_alloc(struct pmap *);
Rewrite tid_flush() in C. There's no need for it to be in asm. Also, by writing in C, and marking it static in pmap.c, it saves a branch to the function itself, as it's only used in one location. The generated asm is virtually identical to the handwritten code.
2015-12-30 02:23:14 +00:00
static void tid_flush(tlbtid_t tid);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
static void tlb_print_entry(int, uint32_t, uint64_t, uint32_t, uint32_t);
#else
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static void tlb_print_entry(int, uint32_t, uint32_t, uint32_t, uint32_t);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
static void tlb1_read_entry(tlb_entry_t *, unsigned int);
static void tlb1_write_entry(tlb_entry_t *, unsigned int);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static int tlb1_iomapped(int, vm_paddr_t, vm_size_t, vm_offset_t *);
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
static vm_size_t tlb1_mapin_region(vm_offset_t, vm_paddr_t, vm_size_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static vm_size_t tsize2size(unsigned int);
static unsigned int size2tsize(vm_size_t);
static unsigned int ilog2(unsigned int);
static void set_mas4_defaults(void);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static inline void tlb0_flush_entry(vm_offset_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static inline unsigned int tlb0_tableidx(vm_offset_t, unsigned int);
/**************************************************************************/
/* Page table management */
/**************************************************************************/
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
static struct rwlock_padalign pvh_global_lock;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Data for the pv entry allocation mechanism */
static uma_zone_t pvzone;
static int pv_entry_count = 0, pv_entry_max = 0, pv_entry_high_water = 0;
#define PV_ENTRY_ZONE_MIN 2048 /* min pv entries in uma zone */
#ifndef PMAP_SHPGPERPROC
#define PMAP_SHPGPERPROC 200
#endif
static void ptbl_init(void);
static struct ptbl_buf *ptbl_buf_alloc(void);
static void ptbl_buf_free(struct ptbl_buf *);
static void ptbl_free_pmap_ptbl(pmap_t, pte_t *);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
static pte_t *ptbl_alloc(mmu_t, pmap_t, pte_t **,
unsigned int, boolean_t);
static void ptbl_free(mmu_t, pmap_t, pte_t **, unsigned int);
static void ptbl_hold(mmu_t, pmap_t, pte_t **, unsigned int);
static int ptbl_unhold(mmu_t, pmap_t, vm_offset_t);
#else
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
static pte_t *ptbl_alloc(mmu_t, pmap_t, unsigned int, boolean_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void ptbl_free(mmu_t, pmap_t, unsigned int);
static void ptbl_hold(mmu_t, pmap_t, unsigned int);
static int ptbl_unhold(mmu_t, pmap_t, unsigned int);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static vm_paddr_t pte_vatopa(mmu_t, pmap_t, vm_offset_t);
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
static int pte_enter(mmu_t, pmap_t, vm_page_t, vm_offset_t, uint32_t, boolean_t);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static int pte_remove(mmu_t, pmap_t, vm_offset_t, uint8_t);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
static pte_t *pte_find(mmu_t, pmap_t, vm_offset_t);
static void kernel_pte_alloc(vm_offset_t, vm_offset_t, vm_offset_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static pv_entry_t pv_alloc(void);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void pv_free(pv_entry_t);
static void pv_insert(pmap_t, vm_offset_t, vm_page_t);
static void pv_remove(pmap_t, vm_offset_t, vm_page_t);
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
static void booke_pmap_init_qpages(void);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Number of kva ptbl buffers, each covering one ptbl (PTBL_PAGES). */
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
#define PTBL_BUFS (16UL * 16 * 16)
#else
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#define PTBL_BUFS (128 * 16)
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
struct ptbl_buf {
TAILQ_ENTRY(ptbl_buf) link; /* list link */
vm_offset_t kva; /* va of mapping */
};
/* ptbl free list and a lock used for access synchronization. */
static TAILQ_HEAD(, ptbl_buf) ptbl_buf_freelist;
static struct mtx ptbl_buf_freelist_lock;
/* Base address of kva space allocated fot ptbl bufs. */
static vm_offset_t ptbl_buf_pool_vabase;
/* Pointer to ptbl_buf structures. */
static struct ptbl_buf *ptbl_bufs;
Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and building without SMP causes a build failure. MFC after: 1 month
2015-03-28 21:39:42 +00:00
#ifdef SMP
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
extern tlb_entry_t __boot_tlb1[];
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
void pmap_bootstrap_ap(volatile uint32_t *);
Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and building without SMP causes a build failure. MFC after: 1 month
2015-03-28 21:39:42 +00:00
#endif
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Kernel MMU interface
*/
static void mmu_booke_clear_modify(mmu_t, vm_page_t);
Fix Book-E/MPC85XX build. Some prototypes were wrong and got revealed with the recent kobj signature checking.
2009-06-13 08:57:04 +00:00
static void mmu_booke_copy(mmu_t, pmap_t, pmap_t, vm_offset_t,
vm_size_t, vm_offset_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void mmu_booke_copy_page(mmu_t, vm_page_t, vm_page_t);
Add pmap function pmap_copy_pages(), which copies the content of the pages around, taking array of vm_page_t both for source and destination. Starting offsets and total transfer size are specified. The function implements optimal algorithm for copying using the platform-specific optimizations. For instance, on the architectures were the direct map is available, no transient mappings are created, for i386 the per-cpu ephemeral page frame is used. The code was typically borrowed from the pmap_copy_page() for the same architecture. Only i386/amd64, powerpc aim and arm/arm-v6 implementations were tested at the time of commit. High-level code, not committed yet to the tree, ensures that the use of the function is only allowed after explicit enablement. For sparc64, the existing code has known issues and a stab is added instead, to allow the kernel linking. Sponsored by: The FreeBSD Foundation Tested by: pho (i386, amd64), scottl (amd64), ian (arm and arm-v6) MFC after: 2 weeks
2013-03-14 20:18:12 +00:00
static void mmu_booke_copy_pages(mmu_t, vm_page_t *,
vm_offset_t, vm_page_t *, vm_offset_t, int);
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
static int mmu_booke_enter(mmu_t, pmap_t, vm_offset_t, vm_page_t,
vm_prot_t, u_int flags, int8_t psind);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void mmu_booke_enter_object(mmu_t, pmap_t, vm_offset_t, vm_offset_t,
vm_page_t, vm_prot_t);
static void mmu_booke_enter_quick(mmu_t, pmap_t, vm_offset_t, vm_page_t,
vm_prot_t);
static vm_paddr_t mmu_booke_extract(mmu_t, pmap_t, vm_offset_t);
static vm_page_t mmu_booke_extract_and_hold(mmu_t, pmap_t, vm_offset_t,
vm_prot_t);
static void mmu_booke_init(mmu_t);
static boolean_t mmu_booke_is_modified(mmu_t, vm_page_t);
static boolean_t mmu_booke_is_prefaultable(mmu_t, pmap_t, vm_offset_t);
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
static boolean_t mmu_booke_is_referenced(mmu_t, vm_page_t);
Avoid recursion on the pvh global lock in the aim oea pmap. Correct the return type of the pmap_ts_referenced() implementations. Reported by: jhibbits [1] Tested by: andreast
2012-07-10 22:10:21 +00:00
static int mmu_booke_ts_referenced(mmu_t, vm_page_t);
Fix physical address type to vm_paddr_t.
2012-05-24 21:13:24 +00:00
static vm_offset_t mmu_booke_map(mmu_t, vm_offset_t *, vm_paddr_t, vm_paddr_t,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
int);
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
static int mmu_booke_mincore(mmu_t, pmap_t, vm_offset_t,
vm_paddr_t *);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void mmu_booke_object_init_pt(mmu_t, pmap_t, vm_offset_t,
vm_object_t, vm_pindex_t, vm_size_t);
static boolean_t mmu_booke_page_exists_quick(mmu_t, pmap_t, vm_page_t);
static void mmu_booke_page_init(mmu_t, vm_page_t);
static int mmu_booke_page_wired_mappings(mmu_t, vm_page_t);
static void mmu_booke_pinit(mmu_t, pmap_t);
static void mmu_booke_pinit0(mmu_t, pmap_t);
static void mmu_booke_protect(mmu_t, pmap_t, vm_offset_t, vm_offset_t,
vm_prot_t);
static void mmu_booke_qenter(mmu_t, vm_offset_t, vm_page_t *, int);
static void mmu_booke_qremove(mmu_t, vm_offset_t, int);
static void mmu_booke_release(mmu_t, pmap_t);
static void mmu_booke_remove(mmu_t, pmap_t, vm_offset_t, vm_offset_t);
static void mmu_booke_remove_all(mmu_t, vm_page_t);
static void mmu_booke_remove_write(mmu_t, vm_page_t);
Implement pmap_unwire(). See r268327 for the motivation behind this change.
2014-07-13 16:27:57 +00:00
static void mmu_booke_unwire(mmu_t, pmap_t, vm_offset_t, vm_offset_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void mmu_booke_zero_page(mmu_t, vm_page_t);
static void mmu_booke_zero_page_area(mmu_t, vm_page_t, int, int);
static void mmu_booke_activate(mmu_t, struct thread *);
static void mmu_booke_deactivate(mmu_t, struct thread *);
static void mmu_booke_bootstrap(mmu_t, vm_offset_t, vm_offset_t);
Fix physical address type to vm_paddr_t.
2012-05-24 21:13:24 +00:00
static void *mmu_booke_mapdev(mmu_t, vm_paddr_t, vm_size_t);
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
static void *mmu_booke_mapdev_attr(mmu_t, vm_paddr_t, vm_size_t, vm_memattr_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void mmu_booke_unmapdev(mmu_t, vm_offset_t, vm_size_t);
Fix physical address type to vm_paddr_t.
2012-05-24 21:13:24 +00:00
static vm_paddr_t mmu_booke_kextract(mmu_t, vm_offset_t);
static void mmu_booke_kenter(mmu_t, vm_offset_t, vm_paddr_t);
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
static void mmu_booke_kenter_attr(mmu_t, vm_offset_t, vm_paddr_t, vm_memattr_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void mmu_booke_kremove(mmu_t, vm_offset_t);
Fix physical address type to vm_paddr_t.
2012-05-24 21:13:24 +00:00
static boolean_t mmu_booke_dev_direct_mapped(mmu_t, vm_paddr_t, vm_size_t);
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked). The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
2009-10-21 18:38:02 +00:00
static void mmu_booke_sync_icache(mmu_t, pmap_t, vm_offset_t,
vm_size_t);
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
static void mmu_booke_dumpsys_map(mmu_t, vm_paddr_t pa, size_t,
void **);
static void mmu_booke_dumpsys_unmap(mmu_t, vm_paddr_t pa, size_t,
void *);
static void mmu_booke_scan_init(mmu_t);
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
static vm_offset_t mmu_booke_quick_enter_page(mmu_t mmu, vm_page_t m);
static void mmu_booke_quick_remove_page(mmu_t mmu, vm_offset_t addr);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
static int mmu_booke_change_attr(mmu_t mmu, vm_offset_t addr,
vm_size_t sz, vm_memattr_t mode);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static mmu_method_t mmu_booke_methods[] = {
/* pmap dispatcher interface */
MMUMETHOD(mmu_clear_modify, mmu_booke_clear_modify),
MMUMETHOD(mmu_copy, mmu_booke_copy),
MMUMETHOD(mmu_copy_page, mmu_booke_copy_page),
Add pmap function pmap_copy_pages(), which copies the content of the pages around, taking array of vm_page_t both for source and destination. Starting offsets and total transfer size are specified. The function implements optimal algorithm for copying using the platform-specific optimizations. For instance, on the architectures were the direct map is available, no transient mappings are created, for i386 the per-cpu ephemeral page frame is used. The code was typically borrowed from the pmap_copy_page() for the same architecture. Only i386/amd64, powerpc aim and arm/arm-v6 implementations were tested at the time of commit. High-level code, not committed yet to the tree, ensures that the use of the function is only allowed after explicit enablement. For sparc64, the existing code has known issues and a stab is added instead, to allow the kernel linking. Sponsored by: The FreeBSD Foundation Tested by: pho (i386, amd64), scottl (amd64), ian (arm and arm-v6) MFC after: 2 weeks
2013-03-14 20:18:12 +00:00
MMUMETHOD(mmu_copy_pages, mmu_booke_copy_pages),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
MMUMETHOD(mmu_enter, mmu_booke_enter),
MMUMETHOD(mmu_enter_object, mmu_booke_enter_object),
MMUMETHOD(mmu_enter_quick, mmu_booke_enter_quick),
MMUMETHOD(mmu_extract, mmu_booke_extract),
MMUMETHOD(mmu_extract_and_hold, mmu_booke_extract_and_hold),
MMUMETHOD(mmu_init, mmu_booke_init),
MMUMETHOD(mmu_is_modified, mmu_booke_is_modified),
MMUMETHOD(mmu_is_prefaultable, mmu_booke_is_prefaultable),
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
MMUMETHOD(mmu_is_referenced, mmu_booke_is_referenced),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
MMUMETHOD(mmu_ts_referenced, mmu_booke_ts_referenced),
MMUMETHOD(mmu_map, mmu_booke_map),
MMUMETHOD(mmu_mincore, mmu_booke_mincore),
MMUMETHOD(mmu_object_init_pt, mmu_booke_object_init_pt),
MMUMETHOD(mmu_page_exists_quick,mmu_booke_page_exists_quick),
MMUMETHOD(mmu_page_init, mmu_booke_page_init),
MMUMETHOD(mmu_page_wired_mappings, mmu_booke_page_wired_mappings),
MMUMETHOD(mmu_pinit, mmu_booke_pinit),
MMUMETHOD(mmu_pinit0, mmu_booke_pinit0),
MMUMETHOD(mmu_protect, mmu_booke_protect),
MMUMETHOD(mmu_qenter, mmu_booke_qenter),
MMUMETHOD(mmu_qremove, mmu_booke_qremove),
MMUMETHOD(mmu_release, mmu_booke_release),
MMUMETHOD(mmu_remove, mmu_booke_remove),
MMUMETHOD(mmu_remove_all, mmu_booke_remove_all),
MMUMETHOD(mmu_remove_write, mmu_booke_remove_write),
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked). The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
2009-10-21 18:38:02 +00:00
MMUMETHOD(mmu_sync_icache, mmu_booke_sync_icache),
Implement pmap_unwire(). See r268327 for the motivation behind this change.
2014-07-13 16:27:57 +00:00
MMUMETHOD(mmu_unwire, mmu_booke_unwire),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
MMUMETHOD(mmu_zero_page, mmu_booke_zero_page),
MMUMETHOD(mmu_zero_page_area, mmu_booke_zero_page_area),
MMUMETHOD(mmu_activate, mmu_booke_activate),
MMUMETHOD(mmu_deactivate, mmu_booke_deactivate),
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
MMUMETHOD(mmu_quick_enter_page, mmu_booke_quick_enter_page),
MMUMETHOD(mmu_quick_remove_page, mmu_booke_quick_remove_page),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Internal interfaces */
MMUMETHOD(mmu_bootstrap, mmu_booke_bootstrap),
MMUMETHOD(mmu_dev_direct_mapped,mmu_booke_dev_direct_mapped),
MMUMETHOD(mmu_mapdev, mmu_booke_mapdev),
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
MMUMETHOD(mmu_mapdev_attr, mmu_booke_mapdev_attr),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
MMUMETHOD(mmu_kenter, mmu_booke_kenter),
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
MMUMETHOD(mmu_kenter_attr, mmu_booke_kenter_attr),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
MMUMETHOD(mmu_kextract, mmu_booke_kextract),
Add missing pmap_kenter() method for book-e. This isn't added to AIM yet, because it's not yet needed. It's needed for Book-E ePAPR boot support. X-MFC With: r304047
2016-08-13 18:57:14 +00:00
MMUMETHOD(mmu_kremove, mmu_booke_kremove),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
MMUMETHOD(mmu_unmapdev, mmu_booke_unmapdev),
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
MMUMETHOD(mmu_change_attr, mmu_booke_change_attr),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
/* dumpsys() support */
MMUMETHOD(mmu_dumpsys_map, mmu_booke_dumpsys_map),
MMUMETHOD(mmu_dumpsys_unmap, mmu_booke_dumpsys_unmap),
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
MMUMETHOD(mmu_scan_init, mmu_booke_scan_init),
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{ 0, 0 }
};
Introduce inheritance into the PowerPC MMU kobj interface. include/mmuvar.h - Change the MMU_DEF macro to also create the class definition as well as define the DATA_SET. Add a macro, MMU_DEF_INHERIT, which has an extra parameter specifying the MMU class to inherit methods from. Update the comments at the start of the header file to describe the new macros. booke/pmap.c aim/mmu_oea.c aim/mmu_oea64.c - Collapse mmu_def_t declaration into updated MMU_DEF macro The MMU_DEF_INHERIT macro will be used in the PS3 MMU implementation to allow it to inherit the stock powerpc64 MMU methods. Reviewed by: nwhitehorn
2010-09-15 00:17:52 +00:00
MMU_DEF(booke_mmu, MMU_TYPE_BOOKE, mmu_booke_methods, 0);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
static __inline uint32_t
Use the correct type for physical addresses. On Book-E, physical addresses are actually 36-bits, not 32-bits. This is currently worked around by ignoring the top bits. However, in some cases, the boot loader configures CCSR to something above the 32-bit mark. This is stage 1 in updating the pmap to handle 36-bit physaddr.
2015-07-04 19:00:38 +00:00
tlb_calc_wimg(vm_paddr_t pa, vm_memattr_t ma)
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
{
uint32_t attrib;
int i;
if (ma != VM_MEMATTR_DEFAULT) {
switch (ma) {
case VM_MEMATTR_UNCACHEABLE:
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
return (MAS2_I | MAS2_G);
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
case VM_MEMATTR_WRITE_COMBINING:
case VM_MEMATTR_WRITE_BACK:
case VM_MEMATTR_PREFETCHABLE:
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
return (MAS2_I);
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
case VM_MEMATTR_WRITE_THROUGH:
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
return (MAS2_W | MAS2_M);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
case VM_MEMATTR_CACHEABLE:
return (MAS2_M);
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
}
}
/*
* Assume the page is cache inhibited and access is guarded unless
* it's in our available memory array.
*/
attrib = _TLB_ENTRY_IO;
for (i = 0; i < physmem_regions_sz; i++) {
if ((pa >= physmem_regions[i].mr_start) &&
(pa < (physmem_regions[i].mr_start +
physmem_regions[i].mr_size))) {
attrib = _TLB_ENTRY_MEM;
break;
}
}
return (attrib);
}
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
static inline void
tlb_miss_lock(void)
{
#ifdef SMP
struct pcpu *pc;
if (!smp_started)
return;
On multi-core, multi-threaded PPC systems, it is important that the threads be brought up in the order they are enumerated in the device tree (in particular, that thread 0 on each core be brought up first). The SLIST through which we loop to start the CPUs has all of its entries added with SLIST_INSERT_HEAD(), which means it is in reverse order of enumeration and so AP startup would always fail in such situations (causing a machine check or RTAS failure). Fix this by changing the SLIST into an STAILQ, and inserting new CPUs at the end. Reviewed by: jhb
2011-05-31 15:11:43 +00:00
STAILQ_FOREACH(pc, &cpuhead, pc_allcpu) {
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
if (pc != pcpup) {
CTR3(KTR_PMAP, "%s: tlb miss LOCK of CPU=%d, "
"tlb_lock=%p", __func__, pc->pc_cpuid, pc->pc_booke_tlb_lock);
KASSERT((pc->pc_cpuid != PCPU_GET(cpuid)),
("tlb_miss_lock: tried to lock self"));
tlb_lock(pc->pc_booke_tlb_lock);
CTR1(KTR_PMAP, "%s: locked", __func__);
}
}
#endif
}
static inline void
tlb_miss_unlock(void)
{
#ifdef SMP
struct pcpu *pc;
if (!smp_started)
return;
On multi-core, multi-threaded PPC systems, it is important that the threads be brought up in the order they are enumerated in the device tree (in particular, that thread 0 on each core be brought up first). The SLIST through which we loop to start the CPUs has all of its entries added with SLIST_INSERT_HEAD(), which means it is in reverse order of enumeration and so AP startup would always fail in such situations (causing a machine check or RTAS failure). Fix this by changing the SLIST into an STAILQ, and inserting new CPUs at the end. Reviewed by: jhb
2011-05-31 15:11:43 +00:00
STAILQ_FOREACH(pc, &cpuhead, pc_allcpu) {
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
if (pc != pcpup) {
CTR2(KTR_PMAP, "%s: tlb miss UNLOCK of CPU=%d",
__func__, pc->pc_cpuid);
tlb_unlock(pc->pc_booke_tlb_lock);
CTR1(KTR_PMAP, "%s: unlocked", __func__);
}
}
#endif
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Return number of entries in TLB0. */
static __inline void
tlb0_get_tlbconf(void)
{
uint32_t tlb0_cfg;
tlb0_cfg = mfspr(SPR_TLB0CFG);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_entries = tlb0_cfg & TLBCFG_NENTRY_MASK;
tlb0_ways = (tlb0_cfg & TLBCFG_ASSOC_MASK) >> TLBCFG_ASSOC_SHIFT;
tlb0_entries_per_way = tlb0_entries / tlb0_ways;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
/* Return number of entries in TLB1. */
static __inline void
tlb1_get_tlbconf(void)
{
uint32_t tlb1_cfg;
tlb1_cfg = mfspr(SPR_TLB1CFG);
tlb1_entries = tlb1_cfg & TLBCFG_NENTRY_MASK;
}
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
/**************************************************************************/
/* Page table related */
/**************************************************************************/
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
/* Initialize pool of kva ptbl buffers. */
static void
ptbl_init(void)
{
int i;
mtx_init(&ptbl_buf_freelist_lock, "ptbl bufs lock", NULL, MTX_DEF);
TAILQ_INIT(&ptbl_buf_freelist);
for (i = 0; i < PTBL_BUFS; i++) {
ptbl_bufs[i].kva = ptbl_buf_pool_vabase +
i * MAX(PTBL_PAGES,PDIR_PAGES) * PAGE_SIZE;
TAILQ_INSERT_TAIL(&ptbl_buf_freelist, &ptbl_bufs[i], link);
}
}
/* Get an sf_buf from the freelist. */
static struct ptbl_buf *
ptbl_buf_alloc(void)
{
struct ptbl_buf *buf;
mtx_lock(&ptbl_buf_freelist_lock);
buf = TAILQ_FIRST(&ptbl_buf_freelist);
if (buf != NULL)
TAILQ_REMOVE(&ptbl_buf_freelist, buf, link);
mtx_unlock(&ptbl_buf_freelist_lock);
return (buf);
}
/* Return ptbl buff to free pool. */
static void
ptbl_buf_free(struct ptbl_buf *buf)
{
mtx_lock(&ptbl_buf_freelist_lock);
TAILQ_INSERT_TAIL(&ptbl_buf_freelist, buf, link);
mtx_unlock(&ptbl_buf_freelist_lock);
}
/*
* Search the list of allocated ptbl bufs and find on list of allocated ptbls
*/
static void
ptbl_free_pmap_ptbl(pmap_t pmap, pte_t * ptbl)
{
struct ptbl_buf *pbuf;
TAILQ_FOREACH(pbuf, &pmap->pm_ptbl_list, link) {
if (pbuf->kva == (vm_offset_t) ptbl) {
/* Remove from pmap ptbl buf list. */
TAILQ_REMOVE(&pmap->pm_ptbl_list, pbuf, link);
/* Free corresponding ptbl buf. */
ptbl_buf_free(pbuf);
break;
}
}
}
/* Get a pointer to a PTE in a page table. */
static __inline pte_t *
pte_find(mmu_t mmu, pmap_t pmap, vm_offset_t va)
{
pte_t **pdir;
pte_t *ptbl;
KASSERT((pmap != NULL), ("pte_find: invalid pmap"));
pdir = pmap->pm_pp2d[PP2D_IDX(va)];
if (!pdir)
return NULL;
ptbl = pdir[PDIR_IDX(va)];
return ((ptbl != NULL) ? &ptbl[PTBL_IDX(va)] : NULL);
}
/*
* Search the list of allocated pdir bufs and find on list of allocated pdirs
*/
static void
ptbl_free_pmap_pdir(mmu_t mmu, pmap_t pmap, pte_t ** pdir)
{
struct ptbl_buf *pbuf;
TAILQ_FOREACH(pbuf, &pmap->pm_pdir_list, link) {
if (pbuf->kva == (vm_offset_t) pdir) {
/* Remove from pmap ptbl buf list. */
TAILQ_REMOVE(&pmap->pm_pdir_list, pbuf, link);
/* Free corresponding pdir buf. */
ptbl_buf_free(pbuf);
break;
}
}
}
/* Free pdir pages and invalidate pdir entry. */
static void
pdir_free(mmu_t mmu, pmap_t pmap, unsigned int pp2d_idx)
{
pte_t **pdir;
vm_paddr_t pa;
vm_offset_t va;
vm_page_t m;
int i;
pdir = pmap->pm_pp2d[pp2d_idx];
KASSERT((pdir != NULL), ("pdir_free: null pdir"));
pmap->pm_pp2d[pp2d_idx] = NULL;
for (i = 0; i < PDIR_PAGES; i++) {
va = ((vm_offset_t) pdir + (i * PAGE_SIZE));
pa = pte_vatopa(mmu, kernel_pmap, va);
m = PHYS_TO_VM_PAGE(pa);
vm_page_free_zero(m);
atomic_subtract_int(&vm_cnt.v_wire_count, 1);
pmap_kremove(va);
}
ptbl_free_pmap_pdir(mmu, pmap, pdir);
}
/*
* Decrement pdir pages hold count and attempt to free pdir pages. Called
* when removing directory entry from pdir.
*
* Return 1 if pdir pages were freed.
*/
static int
pdir_unhold(mmu_t mmu, pmap_t pmap, u_int pp2d_idx)
{
pte_t **pdir;
vm_paddr_t pa;
vm_page_t m;
int i;
KASSERT((pmap != kernel_pmap),
("pdir_unhold: unholding kernel pdir!"));
pdir = pmap->pm_pp2d[pp2d_idx];
KASSERT(((vm_offset_t) pdir >= VM_MIN_KERNEL_ADDRESS),
("pdir_unhold: non kva pdir"));
/* decrement hold count */
for (i = 0; i < PDIR_PAGES; i++) {
pa = pte_vatopa(mmu, kernel_pmap,
(vm_offset_t) pdir + (i * PAGE_SIZE));
m = PHYS_TO_VM_PAGE(pa);
m->wire_count--;
}
/*
* Free pdir pages if there are no dir entries in this pdir.
* wire_count has the same value for all ptbl pages, so check the
* last page.
*/
if (m->wire_count == 0) {
pdir_free(mmu, pmap, pp2d_idx);
return (1);
}
return (0);
}
/*
* Increment hold count for pdir pages. This routine is used when new ptlb
* entry is being inserted into pdir.
*/
static void
pdir_hold(mmu_t mmu, pmap_t pmap, pte_t ** pdir)
{
vm_paddr_t pa;
vm_page_t m;
int i;
KASSERT((pmap != kernel_pmap),
("pdir_hold: holding kernel pdir!"));
KASSERT((pdir != NULL), ("pdir_hold: null pdir"));
for (i = 0; i < PDIR_PAGES; i++) {
pa = pte_vatopa(mmu, kernel_pmap,
(vm_offset_t) pdir + (i * PAGE_SIZE));
m = PHYS_TO_VM_PAGE(pa);
m->wire_count++;
}
}
/* Allocate page table. */
static pte_t *
ptbl_alloc(mmu_t mmu, pmap_t pmap, pte_t ** pdir, unsigned int pdir_idx,
boolean_t nosleep)
{
vm_page_t mtbl [PTBL_PAGES];
vm_page_t m;
struct ptbl_buf *pbuf;
unsigned int pidx;
pte_t *ptbl;
int i, j;
int req;
KASSERT((pdir[pdir_idx] == NULL),
("%s: valid ptbl entry exists!", __func__));
pbuf = ptbl_buf_alloc();
if (pbuf == NULL)
panic("%s: couldn't alloc kernel virtual memory", __func__);
ptbl = (pte_t *) pbuf->kva;
for (i = 0; i < PTBL_PAGES; i++) {
pidx = (PTBL_PAGES * pdir_idx) + i;
req = VM_ALLOC_NOOBJ | VM_ALLOC_WIRED;
while ((m = vm_page_alloc(NULL, pidx, req)) == NULL) {
PMAP_UNLOCK(pmap);
rw_wunlock(&pvh_global_lock);
if (nosleep) {
ptbl_free_pmap_ptbl(pmap, ptbl);
for (j = 0; j < i; j++)
vm_page_free(mtbl[j]);
atomic_subtract_int(&vm_cnt.v_wire_count, i);
return (NULL);
}
VM_WAIT;
rw_wlock(&pvh_global_lock);
PMAP_LOCK(pmap);
}
mtbl[i] = m;
}
/* Mapin allocated pages into kernel_pmap. */
mmu_booke_qenter(mmu, (vm_offset_t) ptbl, mtbl, PTBL_PAGES);
/* Zero whole ptbl. */
bzero((caddr_t) ptbl, PTBL_PAGES * PAGE_SIZE);
/* Add pbuf to the pmap ptbl bufs list. */
TAILQ_INSERT_TAIL(&pmap->pm_ptbl_list, pbuf, link);
return (ptbl);
}
/* Free ptbl pages and invalidate pdir entry. */
static void
ptbl_free(mmu_t mmu, pmap_t pmap, pte_t ** pdir, unsigned int pdir_idx)
{
pte_t *ptbl;
vm_paddr_t pa;
vm_offset_t va;
vm_page_t m;
int i;
ptbl = pdir[pdir_idx];
KASSERT((ptbl != NULL), ("ptbl_free: null ptbl"));
pdir[pdir_idx] = NULL;
for (i = 0; i < PTBL_PAGES; i++) {
va = ((vm_offset_t) ptbl + (i * PAGE_SIZE));
pa = pte_vatopa(mmu, kernel_pmap, va);
m = PHYS_TO_VM_PAGE(pa);
vm_page_free_zero(m);
atomic_subtract_int(&vm_cnt.v_wire_count, 1);
pmap_kremove(va);
}
ptbl_free_pmap_ptbl(pmap, ptbl);
}
/*
* Decrement ptbl pages hold count and attempt to free ptbl pages. Called
* when removing pte entry from ptbl.
*
* Return 1 if ptbl pages were freed.
*/
static int
ptbl_unhold(mmu_t mmu, pmap_t pmap, vm_offset_t va)
{
pte_t *ptbl;
vm_paddr_t pa;
vm_page_t m;
u_int pp2d_idx;
pte_t **pdir;
u_int pdir_idx;
int i;
pp2d_idx = PP2D_IDX(va);
pdir_idx = PDIR_IDX(va);
KASSERT((pmap != kernel_pmap),
("ptbl_unhold: unholding kernel ptbl!"));
pdir = pmap->pm_pp2d[pp2d_idx];
ptbl = pdir[pdir_idx];
KASSERT(((vm_offset_t) ptbl >= VM_MIN_KERNEL_ADDRESS),
("ptbl_unhold: non kva ptbl"));
/* decrement hold count */
for (i = 0; i < PTBL_PAGES; i++) {
pa = pte_vatopa(mmu, kernel_pmap,
(vm_offset_t) ptbl + (i * PAGE_SIZE));
m = PHYS_TO_VM_PAGE(pa);
m->wire_count--;
}
/*
* Free ptbl pages if there are no pte entries in this ptbl.
* wire_count has the same value for all ptbl pages, so check the
* last page.
*/
if (m->wire_count == 0) {
/* A pair of indirect entries might point to this ptbl page */
#if 0
tlb_flush_entry(pmap, va & ~((2UL * PAGE_SIZE_1M) - 1),
TLB_SIZE_1M, MAS6_SIND);
tlb_flush_entry(pmap, (va & ~((2UL * PAGE_SIZE_1M) - 1)) | PAGE_SIZE_1M,
TLB_SIZE_1M, MAS6_SIND);
#endif
ptbl_free(mmu, pmap, pdir, pdir_idx);
pdir_unhold(mmu, pmap, pp2d_idx);
return (1);
}
return (0);
}
/*
* Increment hold count for ptbl pages. This routine is used when new pte
* entry is being inserted into ptbl.
*/
static void
ptbl_hold(mmu_t mmu, pmap_t pmap, pte_t ** pdir, unsigned int pdir_idx)
{
vm_paddr_t pa;
pte_t *ptbl;
vm_page_t m;
int i;
KASSERT((pmap != kernel_pmap),
("ptbl_hold: holding kernel ptbl!"));
ptbl = pdir[pdir_idx];
KASSERT((ptbl != NULL), ("ptbl_hold: null ptbl"));
for (i = 0; i < PTBL_PAGES; i++) {
pa = pte_vatopa(mmu, kernel_pmap,
(vm_offset_t) ptbl + (i * PAGE_SIZE));
m = PHYS_TO_VM_PAGE(pa);
m->wire_count++;
}
}
#else
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Initialize pool of kva ptbl buffers. */
static void
ptbl_init(void)
{
int i;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
CTR3(KTR_PMAP, "%s: s (ptbl_bufs = 0x%08x size 0x%08x)", __func__,
(uint32_t)ptbl_bufs, sizeof(struct ptbl_buf) * PTBL_BUFS);
CTR3(KTR_PMAP, "%s: s (ptbl_buf_pool_vabase = 0x%08x size = 0x%08x)",
__func__, ptbl_buf_pool_vabase, PTBL_BUFS * PTBL_PAGES * PAGE_SIZE);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mtx_init(&ptbl_buf_freelist_lock, "ptbl bufs lock", NULL, MTX_DEF);
TAILQ_INIT(&ptbl_buf_freelist);
for (i = 0; i < PTBL_BUFS; i++) {
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
ptbl_bufs[i].kva =
ptbl_buf_pool_vabase + i * PTBL_PAGES * PAGE_SIZE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_INSERT_TAIL(&ptbl_buf_freelist, &ptbl_bufs[i], link);
}
}
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
/* Get a ptbl_buf from the freelist. */
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static struct ptbl_buf *
ptbl_buf_alloc(void)
{
struct ptbl_buf *buf;
mtx_lock(&ptbl_buf_freelist_lock);
buf = TAILQ_FIRST(&ptbl_buf_freelist);
if (buf != NULL)
TAILQ_REMOVE(&ptbl_buf_freelist, buf, link);
mtx_unlock(&ptbl_buf_freelist_lock);
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
CTR2(KTR_PMAP, "%s: buf = %p", __func__, buf);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (buf);
}
/* Return ptbl buff to free pool. */
static void
ptbl_buf_free(struct ptbl_buf *buf)
{
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR2(KTR_PMAP, "%s: buf = %p", __func__, buf);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mtx_lock(&ptbl_buf_freelist_lock);
TAILQ_INSERT_TAIL(&ptbl_buf_freelist, buf, link);
mtx_unlock(&ptbl_buf_freelist_lock);
}
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* Search the list of allocated ptbl bufs and find on list of allocated ptbls
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
static void
ptbl_free_pmap_ptbl(pmap_t pmap, pte_t *ptbl)
{
struct ptbl_buf *pbuf;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR2(KTR_PMAP, "%s: ptbl = %p", __func__, ptbl);
PMAP_LOCK_ASSERT(pmap, MA_OWNED);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
TAILQ_FOREACH(pbuf, &pmap->pm_ptbl_list, link)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (pbuf->kva == (vm_offset_t)ptbl) {
/* Remove from pmap ptbl buf list. */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
TAILQ_REMOVE(&pmap->pm_ptbl_list, pbuf, link);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Free corresponding ptbl buf. */
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
ptbl_buf_free(pbuf);
break;
}
}
/* Allocate page table. */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static pte_t *
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
ptbl_alloc(mmu_t mmu, pmap_t pmap, unsigned int pdir_idx, boolean_t nosleep)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
vm_page_t mtbl[PTBL_PAGES];
vm_page_t m;
struct ptbl_buf *pbuf;
unsigned int pidx;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
pte_t *ptbl;
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
int i, j;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR4(KTR_PMAP, "%s: pmap = %p su = %d pdir_idx = %d", __func__, pmap,
(pmap == kernel_pmap), pdir_idx);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((pdir_idx <= (VM_MAXUSER_ADDRESS / PDIR_SIZE)),
("ptbl_alloc: invalid pdir_idx"));
KASSERT((pmap->pm_pdir[pdir_idx] == NULL),
("pte_alloc: valid ptbl entry exists!"));
pbuf = ptbl_buf_alloc();
if (pbuf == NULL)
panic("pte_alloc: couldn't alloc kernel virtual memory");
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
ptbl = (pte_t *)pbuf->kva;
CTR2(KTR_PMAP, "%s: ptbl kva = %p", __func__, ptbl);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
for (i = 0; i < PTBL_PAGES; i++) {
pidx = (PTBL_PAGES * pdir_idx) + i;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
while ((m = vm_page_alloc(NULL, pidx,
VM_ALLOC_NOOBJ | VM_ALLOC_WIRED)) == NULL) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pmap);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
if (nosleep) {
ptbl_free_pmap_ptbl(pmap, ptbl);
for (j = 0; j < i; j++)
vm_page_free(mtbl[j]);
atomic_subtract_int(&vm_cnt.v_wire_count, i);
return (NULL);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
VM_WAIT;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
}
mtbl[i] = m;
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Map allocated pages into kernel_pmap. */
mmu_booke_qenter(mmu, (vm_offset_t)ptbl, mtbl, PTBL_PAGES);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Zero whole ptbl. */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
bzero((caddr_t)ptbl, PTBL_PAGES * PAGE_SIZE);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Add pbuf to the pmap ptbl bufs list. */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
TAILQ_INSERT_TAIL(&pmap->pm_ptbl_list, pbuf, link);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
return (ptbl);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/* Free ptbl pages and invalidate pdir entry. */
static void
ptbl_free(mmu_t mmu, pmap_t pmap, unsigned int pdir_idx)
{
pte_t *ptbl;
vm_paddr_t pa;
vm_offset_t va;
vm_page_t m;
int i;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR4(KTR_PMAP, "%s: pmap = %p su = %d pdir_idx = %d", __func__, pmap,
(pmap == kernel_pmap), pdir_idx);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((pdir_idx <= (VM_MAXUSER_ADDRESS / PDIR_SIZE)),
("ptbl_free: invalid pdir_idx"));
ptbl = pmap->pm_pdir[pdir_idx];
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR2(KTR_PMAP, "%s: ptbl = %p", __func__, ptbl);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((ptbl != NULL), ("ptbl_free: null ptbl"));
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* Invalidate the pdir entry as soon as possible, so that other CPUs
* don't attempt to look up the page tables we are releasing.
*/
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
pmap->pm_pdir[pdir_idx] = NULL;
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
for (i = 0; i < PTBL_PAGES; i++) {
va = ((vm_offset_t)ptbl + (i * PAGE_SIZE));
pa = pte_vatopa(mmu, kernel_pmap, va);
m = PHYS_TO_VM_PAGE(pa);
vm_page_free_zero(m);
Rename global cnt to vm_cnt to avoid shadowing. To reduce the diff struct pcu.cnt field was not renamed, so PCPU_OP(cnt.field) is still used. pc_cnt and pcpu are also used in kvm(3) and vmstat(8). The goal was to not affect externally used KPI. Bump __FreeBSD_version_ in case some out-of-tree module/code relies on the the global cnt variable. Exp-run revealed no ports using it directly. No objection from: arch@ Sponsored by: EMC / Isilon Storage Division
2014-03-22 10:26:09 +00:00
atomic_subtract_int(&vm_cnt.v_wire_count, 1);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mmu_booke_kremove(mmu, va);
}
ptbl_free_pmap_ptbl(pmap, ptbl);
}
/*
* Decrement ptbl pages hold count and attempt to free ptbl pages.
* Called when removing pte entry from ptbl.
*
* Return 1 if ptbl pages were freed.
*/
static int
ptbl_unhold(mmu_t mmu, pmap_t pmap, unsigned int pdir_idx)
{
pte_t *ptbl;
vm_paddr_t pa;
vm_page_t m;
int i;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
CTR4(KTR_PMAP, "%s: pmap = %p su = %d pdir_idx = %d", __func__, pmap,
(pmap == kernel_pmap), pdir_idx);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((pdir_idx <= (VM_MAXUSER_ADDRESS / PDIR_SIZE)),
("ptbl_unhold: invalid pdir_idx"));
KASSERT((pmap != kernel_pmap),
("ptbl_unhold: unholding kernel ptbl!"));
ptbl = pmap->pm_pdir[pdir_idx];
//debugf("ptbl_unhold: ptbl = 0x%08x\n", (u_int32_t)ptbl);
KASSERT(((vm_offset_t)ptbl >= VM_MIN_KERNEL_ADDRESS),
("ptbl_unhold: non kva ptbl"));
/* decrement hold count */
for (i = 0; i < PTBL_PAGES; i++) {
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
pa = pte_vatopa(mmu, kernel_pmap,
(vm_offset_t)ptbl + (i * PAGE_SIZE));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
m = PHYS_TO_VM_PAGE(pa);
m->wire_count--;
}
/*
* Free ptbl pages if there are no pte etries in this ptbl.
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* wire_count has the same value for all ptbl pages, so check the last
* page.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
if (m->wire_count == 0) {
ptbl_free(mmu, pmap, pdir_idx);
//debugf("ptbl_unhold: e (freed ptbl)\n");
return (1);
}
return (0);
}
/*
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* Increment hold count for ptbl pages. This routine is used when a new pte
* entry is being inserted into the ptbl.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
static void
ptbl_hold(mmu_t mmu, pmap_t pmap, unsigned int pdir_idx)
{
vm_paddr_t pa;
pte_t *ptbl;
vm_page_t m;
int i;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
CTR3(KTR_PMAP, "%s: pmap = %p pdir_idx = %d", __func__, pmap,
pdir_idx);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((pdir_idx <= (VM_MAXUSER_ADDRESS / PDIR_SIZE)),
("ptbl_hold: invalid pdir_idx"));
KASSERT((pmap != kernel_pmap),
("ptbl_hold: holding kernel ptbl!"));
ptbl = pmap->pm_pdir[pdir_idx];
KASSERT((ptbl != NULL), ("ptbl_hold: null ptbl"));
for (i = 0; i < PTBL_PAGES; i++) {
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
pa = pte_vatopa(mmu, kernel_pmap,
(vm_offset_t)ptbl + (i * PAGE_SIZE));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
m = PHYS_TO_VM_PAGE(pa);
m->wire_count++;
}
}
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Allocate pv_entry structure. */
pv_entry_t
pv_alloc(void)
{
pv_entry_t pv;
pv_entry_count++;
Correct the method of waking the page daemon when the number of allocated pv entries surpasses the high water mark. The problem was that the page daemon would only be awakened the first time that the high water mark was surpassed. (The variable "pagedaemon_waken" is a non-working vestige of FreeBSD 4.x, in which it was external and reset by the page daemon whenever it ran. This reset allowed subsequent wakeups by the pv entry allocator.)
2009-06-13 18:35:29 +00:00
if (pv_entry_count > pv_entry_high_water)
pagedaemon_wakeup();
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pv = uma_zalloc(pvzone, M_NOWAIT);
return (pv);
}
/* Free pv_entry structure. */
static __inline void
pv_free(pv_entry_t pve)
{
pv_entry_count--;
uma_zfree(pvzone, pve);
}
/* Allocate and initialize pv_entry structure. */
static void
pv_insert(pmap_t pmap, vm_offset_t va, vm_page_t m)
{
pv_entry_t pve;
//int su = (pmap == kernel_pmap);
//debugf("pv_insert: s (su = %d pmap = 0x%08x va = 0x%08x m = 0x%08x)\n", su,
// (u_int32_t)pmap, va, (u_int32_t)m);
pve = pv_alloc();
if (pve == NULL)
panic("pv_insert: no pv entries!");
pve->pv_pmap = pmap;
pve->pv_va = va;
/* add to pv_list */
PMAP_LOCK_ASSERT(pmap, MA_OWNED);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_assert(&pvh_global_lock, RA_WLOCKED);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_INSERT_TAIL(&m->md.pv_list, pve, pv_link);
//debugf("pv_insert: e\n");
}
/* Destroy pv entry. */
static void
pv_remove(pmap_t pmap, vm_offset_t va, vm_page_t m)
{
pv_entry_t pve;
//int su = (pmap == kernel_pmap);
//debugf("pv_remove: s (su = %d pmap = 0x%08x va = 0x%08x)\n", su, (u_int32_t)pmap, va);
PMAP_LOCK_ASSERT(pmap, MA_OWNED);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_assert(&pvh_global_lock, RA_WLOCKED);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* find pv entry */
TAILQ_FOREACH(pve, &m->md.pv_list, pv_link) {
if ((pmap == pve->pv_pmap) && (va == pve->pv_va)) {
/* remove from pv_list */
TAILQ_REMOVE(&m->md.pv_list, pve, pv_link);
if (TAILQ_EMPTY(&m->md.pv_list))
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_clear(m, PGA_WRITEABLE);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* free pv entry struct */
pv_free(pve);
break;
}
}
//debugf("pv_remove: e\n");
}
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
/*
* Clean pte entry, try to free page table page if requested.
*
* Return 1 if ptbl pages were freed, otherwise return 0.
*/
static int
pte_remove(mmu_t mmu, pmap_t pmap, vm_offset_t va, u_int8_t flags)
{
vm_page_t m;
pte_t *pte;
pte = pte_find(mmu, pmap, va);
KASSERT(pte != NULL, ("%s: NULL pte", __func__));
if (!PTE_ISVALID(pte))
return (0);
/* Get vm_page_t for mapped pte. */
m = PHYS_TO_VM_PAGE(PTE_PA(pte));
if (PTE_ISWIRED(pte))
pmap->pm_stats.wired_count--;
/* Handle managed entry. */
if (PTE_ISMANAGED(pte)) {
/* Handle modified pages. */
if (PTE_ISMODIFIED(pte))
vm_page_dirty(m);
/* Referenced pages. */
if (PTE_ISREFERENCED(pte))
vm_page_aflag_set(m, PGA_REFERENCED);
/* Remove pv_entry from pv_list. */
pv_remove(pmap, va, m);
}
mtx_lock_spin(&tlbivax_mutex);
tlb_miss_lock();
tlb0_flush_entry(va);
*pte = 0;
tlb_miss_unlock();
mtx_unlock_spin(&tlbivax_mutex);
pmap->pm_stats.resident_count--;
if (flags & PTBL_UNHOLD) {
return (ptbl_unhold(mmu, pmap, va));
}
return (0);
}
/*
* allocate a page of pointers to page directories, do not preallocate the
* page tables
*/
static pte_t **
pdir_alloc(mmu_t mmu, pmap_t pmap, unsigned int pp2d_idx, bool nosleep)
{
vm_page_t mtbl [PDIR_PAGES];
vm_page_t m;
struct ptbl_buf *pbuf;
pte_t **pdir;
unsigned int pidx;
int i;
int req;
pbuf = ptbl_buf_alloc();
if (pbuf == NULL)
panic("%s: couldn't alloc kernel virtual memory", __func__);
/* Allocate pdir pages, this will sleep! */
for (i = 0; i < PDIR_PAGES; i++) {
pidx = (PDIR_PAGES * pp2d_idx) + i;
req = VM_ALLOC_NOOBJ | VM_ALLOC_WIRED;
while ((m = vm_page_alloc(NULL, pidx, req)) == NULL) {
PMAP_UNLOCK(pmap);
VM_WAIT;
PMAP_LOCK(pmap);
}
mtbl[i] = m;
}
/* Mapin allocated pages into kernel_pmap. */
pdir = (pte_t **) pbuf->kva;
pmap_qenter((vm_offset_t) pdir, mtbl, PDIR_PAGES);
/* Zero whole pdir. */
bzero((caddr_t) pdir, PDIR_PAGES * PAGE_SIZE);
/* Add pdir to the pmap pdir bufs list. */
TAILQ_INSERT_TAIL(&pmap->pm_pdir_list, pbuf, link);
return pdir;
}
/*
* Insert PTE for a given page and virtual address.
*/
static int
pte_enter(mmu_t mmu, pmap_t pmap, vm_page_t m, vm_offset_t va, uint32_t flags,
boolean_t nosleep)
{
unsigned int pp2d_idx = PP2D_IDX(va);
unsigned int pdir_idx = PDIR_IDX(va);
unsigned int ptbl_idx = PTBL_IDX(va);
pte_t *ptbl, *pte;
pte_t **pdir;
/* Get the page directory pointer. */
pdir = pmap->pm_pp2d[pp2d_idx];
if (pdir == NULL)
pdir = pdir_alloc(mmu, pmap, pp2d_idx, nosleep);
/* Get the page table pointer. */
ptbl = pdir[pdir_idx];
if (ptbl == NULL) {
/* Allocate page table pages. */
ptbl = ptbl_alloc(mmu, pmap, pdir, pdir_idx, nosleep);
if (ptbl == NULL) {
KASSERT(nosleep, ("nosleep and NULL ptbl"));
return (ENOMEM);
}
} else {
/*
* Check if there is valid mapping for requested va, if there
* is, remove it.
*/
pte = &pdir[pdir_idx][ptbl_idx];
if (PTE_ISVALID(pte)) {
pte_remove(mmu, pmap, va, PTBL_HOLD);
} else {
/*
* pte is not used, increment hold count for ptbl
* pages.
*/
if (pmap != kernel_pmap)
ptbl_hold(mmu, pmap, pdir, pdir_idx);
}
}
if (pdir[pdir_idx] == NULL) {
if (pmap != kernel_pmap && pmap->pm_pp2d[pp2d_idx] != NULL)
pdir_hold(mmu, pmap, pdir);
pdir[pdir_idx] = ptbl;
}
if (pmap->pm_pp2d[pp2d_idx] == NULL)
pmap->pm_pp2d[pp2d_idx] = pdir;
/*
* Insert pv_entry into pv_list for mapped page if part of managed
* memory.
*/
if ((m->oflags & VPO_UNMANAGED) == 0) {
flags |= PTE_MANAGED;
/* Create and insert pv entry. */
pv_insert(pmap, va, m);
}
mtx_lock_spin(&tlbivax_mutex);
tlb_miss_lock();
tlb0_flush_entry(va);
pmap->pm_stats.resident_count++;
pte = &pdir[pdir_idx][ptbl_idx];
*pte = PTE_RPN_FROM_PA(VM_PAGE_TO_PHYS(m));
*pte |= (PTE_VALID | flags);
tlb_miss_unlock();
mtx_unlock_spin(&tlbivax_mutex);
return (0);
}
/* Return the pa for the given pmap/va. */
static vm_paddr_t
pte_vatopa(mmu_t mmu, pmap_t pmap, vm_offset_t va)
{
vm_paddr_t pa = 0;
pte_t *pte;
pte = pte_find(mmu, pmap, va);
if ((pte != NULL) && PTE_ISVALID(pte))
pa = (PTE_PA(pte) | (va & PTE_PA_MASK));
return (pa);
}
/* allocate pte entries to manage (addr & mask) to (addr & mask) + size */
static void
kernel_pte_alloc(vm_offset_t data_end, vm_offset_t addr, vm_offset_t pdir)
{
int i, j;
vm_offset_t va;
pte_t *pte;
va = addr;
/* Initialize kernel pdir */
for (i = 0; i < kernel_pdirs; i++) {
kernel_pmap->pm_pp2d[i + PP2D_IDX(va)] =
(pte_t **)(pdir + (i * PAGE_SIZE * PDIR_PAGES));
for (j = PDIR_IDX(va + (i * PAGE_SIZE * PDIR_NENTRIES * PTBL_NENTRIES));
j < PDIR_NENTRIES; j++) {
kernel_pmap->pm_pp2d[i + PP2D_IDX(va)][j] =
(pte_t *)(pdir + (kernel_pdirs * PAGE_SIZE * PDIR_PAGES) +
(((i * PDIR_NENTRIES) + j) * PAGE_SIZE * PTBL_PAGES));
}
}
/*
* Fill in PTEs covering kernel code and data. They are not required
* for address translation, as this area is covered by static TLB1
* entries, but for pte_vatopa() to work correctly with kernel area
* addresses.
*/
for (va = addr; va < data_end; va += PAGE_SIZE) {
pte = &(kernel_pmap->pm_pp2d[PP2D_IDX(va)][PDIR_IDX(va)][PTBL_IDX(va)]);
*pte = PTE_RPN_FROM_PA(kernload + (va - kernstart));
*pte |= PTE_M | PTE_SR | PTE_SW | PTE_SX | PTE_WIRED |
PTE_VALID | PTE_PS_4KB;
}
}
#else
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Clean pte entry, try to free page table page if requested.
*
* Return 1 if ptbl pages were freed, otherwise return 0.
*/
static int
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
pte_remove(mmu_t mmu, pmap_t pmap, vm_offset_t va, uint8_t flags)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
unsigned int pdir_idx = PDIR_IDX(va);
unsigned int ptbl_idx = PTBL_IDX(va);
vm_page_t m;
pte_t *ptbl;
pte_t *pte;
//int su = (pmap == kernel_pmap);
//debugf("pte_remove: s (su = %d pmap = 0x%08x va = 0x%08x flags = %d)\n",
// su, (u_int32_t)pmap, va, flags);
ptbl = pmap->pm_pdir[pdir_idx];
KASSERT(ptbl, ("pte_remove: null ptbl"));
pte = &ptbl[ptbl_idx];
if (pte == NULL || !PTE_ISVALID(pte))
return (0);
if (PTE_ISWIRED(pte))
pmap->pm_stats.wired_count--;
Simplify the page tracking for VA<->PA translations. Drop the tracking down to the pmap layer, with optimizations to only track necessary pages. This should give a (slight) performance improvement, as well as a stability improvement, as the tracking is already mostly handled by the pmap layer.
2016-11-16 05:24:42 +00:00
/* Get vm_page_t for mapped pte. */
m = PHYS_TO_VM_PAGE(PTE_PA(pte));
Remove PTE_ISFAKE. While here remove code between "#if 0" and "#endif".
2009-04-24 02:53:38 +00:00
/* Handle managed entry. */
if (PTE_ISMANAGED(pte)) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Remove PTE_ISFAKE. While here remove code between "#if 0" and "#endif".
2009-04-24 02:53:38 +00:00
if (PTE_ISMODIFIED(pte))
vm_page_dirty(m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Remove PTE_ISFAKE. While here remove code between "#if 0" and "#endif".
2009-04-24 02:53:38 +00:00
if (PTE_ISREFERENCED(pte))
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_set(m, PGA_REFERENCED);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Remove PTE_ISFAKE. While here remove code between "#if 0" and "#endif".
2009-04-24 02:53:38 +00:00
pv_remove(pmap, va, m);
Simplify the page tracking for VA<->PA translations. Drop the tracking down to the pmap layer, with optimizations to only track necessary pages. This should give a (slight) performance improvement, as well as a stability improvement, as the tracking is already mostly handled by the pmap layer.
2016-11-16 05:24:42 +00:00
} else if (m->md.pv_tracked) {
/*
* Always pv_insert()/pv_remove() on MPC85XX, in case DPAA is
* used. This is needed by the NCSW support code for fast
* VA<->PA translation.
*/
pv_remove(pmap, va, m);
if (TAILQ_EMPTY(&m->md.pv_list))
m->md.pv_tracked = false;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(va);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte = 0;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pmap->pm_stats.resident_count--;
if (flags & PTBL_UNHOLD) {
//debugf("pte_remove: e (unhold)\n");
return (ptbl_unhold(mmu, pmap, pdir_idx));
}
//debugf("pte_remove: e\n");
return (0);
}
/*
* Insert PTE for a given page and virtual address.
*/
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
static int
pte_enter(mmu_t mmu, pmap_t pmap, vm_page_t m, vm_offset_t va, uint32_t flags,
boolean_t nosleep)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
unsigned int pdir_idx = PDIR_IDX(va);
unsigned int ptbl_idx = PTBL_IDX(va);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
pte_t *ptbl, *pte;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR4(KTR_PMAP, "%s: su = %d pmap = %p va = %p", __func__,
pmap == kernel_pmap, pmap, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Get the page table pointer. */
ptbl = pmap->pm_pdir[pdir_idx];
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
if (ptbl == NULL) {
/* Allocate page table pages. */
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
ptbl = ptbl_alloc(mmu, pmap, pdir_idx, nosleep);
if (ptbl == NULL) {
KASSERT(nosleep, ("nosleep and NULL ptbl"));
return (ENOMEM);
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
} else {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Check if there is valid mapping for requested
* va, if there is, remove it.
*/
pte = &pmap->pm_pdir[pdir_idx][ptbl_idx];
if (PTE_ISVALID(pte)) {
pte_remove(mmu, pmap, va, PTBL_HOLD);
} else {
/*
* pte is not used, increment hold count
* for ptbl pages.
*/
if (pmap != kernel_pmap)
ptbl_hold(mmu, pmap, pdir_idx);
}
}
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* Insert pv_entry into pv_list for mapped page if part of managed
* memory.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
if ((m->oflags & VPO_UNMANAGED) == 0) {
flags |= PTE_MANAGED;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
/* Create and insert pv entry. */
pv_insert(pmap, va, m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
pmap->pm_stats.resident_count++;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(va);
if (pmap->pm_pdir[pdir_idx] == NULL) {
/*
* If we just allocated a new page table, hook it in
* the pdir.
*/
pmap->pm_pdir[pdir_idx] = ptbl;
}
pte = &(pmap->pm_pdir[pdir_idx][ptbl_idx]);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte = PTE_RPN_FROM_PA(VM_PAGE_TO_PHYS(m));
*pte |= (PTE_VALID | flags | PTE_PS_4KB); /* 4KB pages only */
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
return (0);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/* Return the pa for the given pmap/va. */
static vm_paddr_t
pte_vatopa(mmu_t mmu, pmap_t pmap, vm_offset_t va)
{
vm_paddr_t pa = 0;
pte_t *pte;
pte = pte_find(mmu, pmap, va);
if ((pte != NULL) && PTE_ISVALID(pte))
pa = (PTE_PA(pte) | (va & PTE_PA_MASK));
return (pa);
}
/* Get a pointer to a PTE in a page table. */
static pte_t *
pte_find(mmu_t mmu, pmap_t pmap, vm_offset_t va)
{
unsigned int pdir_idx = PDIR_IDX(va);
unsigned int ptbl_idx = PTBL_IDX(va);
KASSERT((pmap != NULL), ("pte_find: invalid pmap"));
if (pmap->pm_pdir[pdir_idx])
return (&(pmap->pm_pdir[pdir_idx][ptbl_idx]));
return (NULL);
}
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
/* Set up kernel page tables. */
static void
kernel_pte_alloc(vm_offset_t data_end, vm_offset_t addr, vm_offset_t pdir)
{
int i;
vm_offset_t va;
pte_t *pte;
/* Initialize kernel pdir */
for (i = 0; i < kernel_ptbls; i++)
kernel_pmap->pm_pdir[kptbl_min + i] =
(pte_t *)(pdir + (i * PAGE_SIZE * PTBL_PAGES));
/*
* Fill in PTEs covering kernel code and data. They are not required
* for address translation, as this area is covered by static TLB1
* entries, but for pte_vatopa() to work correctly with kernel area
* addresses.
*/
for (va = addr; va < data_end; va += PAGE_SIZE) {
pte = &(kernel_pmap->pm_pdir[PDIR_IDX(va)][PTBL_IDX(va)]);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte = PTE_RPN_FROM_PA(kernload + (va - kernstart));
*pte |= PTE_M | PTE_SR | PTE_SW | PTE_SX | PTE_WIRED |
PTE_VALID | PTE_PS_4KB;
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
}
}
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/**************************************************************************/
/* PMAP related */
/**************************************************************************/
/*
Better support different kernel hand-offs. When loaded directly from U-Boot, the kernel is passed a standard argc/argv pair. The Juniper loader passes the metadata pointer as the second argument and passes 0 in the first. The FreeBSD loader passes the metadata pointer in the first argument. As such, have locore preserve the first 2 arguments in registers r30 & r31. Change e500_init() to accept these arguments. Don't pass global offsets (i.e. kernel_text and _end) as arguments to e500_init(). We can reference those directly. Rename e500_init() to booke_init() now that we're changing the prototype. In booke_init(), "decode" arg1 and arg2 to obtain the metadata pointer correctly. For the U-Boot case, clear SBSS and BSS and bank on having a static FDT for now. This allows loading the ELF kernel and jumping to the entry point without trampoline.
2011-05-28 04:10:44 +00:00
* This is called during booke_init, before the system is really initialized.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
static void
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
mmu_booke_bootstrap(mmu_t mmu, vm_offset_t start, vm_offset_t kernelend)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
vm_paddr_t phys_kernelend;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
struct mem_region *mp, *mp1;
int cnt, i, j;
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
vm_paddr_t s, e, sz;
vm_paddr_t physsz, hwphyssz;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
u_int phys_avail_count;
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
vm_size_t kstack0_sz;
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
vm_offset_t kernel_pdir, kstack0;
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
vm_paddr_t kstack0_phys;
Implement a facility for dynamic per-cpu variables. - Modules and kernel code alike may use DPCPU_DEFINE(), DPCPU_GET(), DPCPU_SET(), etc. akin to the statically defined PCPU_*. Requires only one extra instruction more than PCPU_* and is virtually the same as __thread for builtin and much faster for shared objects. DPCPU variables can be initialized when defined. - Modules are supported by relocating the module's per-cpu linker set over space reserved in the kernel. Modules may fail to load if there is insufficient space available. - Track space available for modules with a one-off extent allocator. Free may block for memory to allocate space for an extent. Reviewed by: jhb, rwatson, kan, sam, grehan, marius, marcel, stas
2009-06-23 22:42:39 +00:00
void *dpcpu;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
debugf("mmu_booke_bootstrap: entered\n");
Initialize NX stack capabilities and direct map status in pmap like on AIM.
2015-03-01 21:23:23 +00:00
/* Set interesting system properties */
hw_direct_map = 0;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#if defined(COMPAT_FREEBSD32) || !defined(__powerpc64__)
Initialize NX stack capabilities and direct map status in pmap like on AIM.
2015-03-01 21:23:23 +00:00
elf32_nxstack = 1;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Initialize NX stack capabilities and direct map status in pmap like on AIM.
2015-03-01 21:23:23 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Initialize invalidation mutex */
mtx_init(&tlbivax_mutex, "tlbivax", NULL, MTX_SPIN);
/* Read TLB0 size and associativity. */
tlb0_get_tlbconf();
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
/*
* Align kernel start and end address (kernel image).
* Note that kernel end does not necessarily relate to kernsize.
* kernsize is the size of the kernel that is actually mapped.
*/
Unify booke and AIM machdep. Much of the code was common to begin with. There is one nit, which is likely not an issue at all. With the old code, the AIM machdep would __syncicache() the entire kernel core at setup. However, in the unified setup, that seems to hang on the MPC7455, perhaps because it's running later than before. Removing this allows it to boot just fine. Examining the code, the FreeBSD loader already does syncicache of the full kernel, and each module loaded, so this doesn't appear to be an actual problem. Initial code by Nathan Whitehorn.
2015-04-30 01:24:25 +00:00
kernstart = trunc_page(start);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
data_start = round_page(kernelend);
data_end = data_start;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
/*
* Addresses of preloaded modules (like file systems) use
* physical addresses. Make sure we relocate those into
* virtual addresses.
*/
preload_addr_relocate = kernstart - kernload;
/* Allocate the dynamic per-cpu area. */
dpcpu = (void *)data_end;
data_end += DPCPU_SIZE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Allocate space for the message buffer. */
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
msgbufp = (struct msgbuf *)data_end;
Make MSGBUF_SIZE kernel option a loader tunable kern.msgbufsize. Submitted by: perryh pluto.rain.com (previous version) Reviewed by: jhb Approved by: kib (mentor) Tested by: universe
2011-01-21 10:26:26 +00:00
data_end += msgbufsize;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf(" msgbufp at 0x%"PRI0ptrX" end = 0x%"PRI0ptrX"\n",
(uintptr_t)msgbufp, data_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
data_end = round_page(data_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Allocate space for ptbl_bufs. */
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
ptbl_bufs = (struct ptbl_buf *)data_end;
data_end += sizeof(struct ptbl_buf) * PTBL_BUFS;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf(" ptbl_bufs at 0x%"PRI0ptrX" end = 0x%"PRI0ptrX"\n",
(uintptr_t)ptbl_bufs, data_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
data_end = round_page(data_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Allocate PTE tables for kernel KVA. */
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
kernel_pdir = data_end;
sys/powerpc: make use of the howmany() macro when available. We have a howmany() macro in the <sys/param.h> header that is convenient to re-use as it makes things easier to read.
2016-04-26 14:44:49 +00:00
kernel_ptbls = howmany(VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS,
PDIR_SIZE);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
kernel_pdirs = howmany(kernel_ptbls, PDIR_NENTRIES);
data_end += kernel_pdirs * PDIR_PAGES * PAGE_SIZE;
#endif
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
data_end += kernel_ptbls * PTBL_PAGES * PAGE_SIZE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
debugf(" kernel ptbls: %d\n", kernel_ptbls);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf(" kernel pdir at 0x%"PRI0ptrX" end = 0x%"PRI0ptrX"\n",
kernel_pdir, data_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf(" data_end: 0x%"PRI0ptrX"\n", data_end);
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
if (data_end - kernstart > kernsize) {
kernsize += tlb1_mapin_region(kernstart + kernsize,
kernload + kernsize, (data_end - kernstart) - kernsize);
}
data_end = kernstart + kernsize;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf(" updated data_end: 0x%"PRI0ptrX"\n", data_end);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
/*
* Clear the structures - note we can only do it safely after the
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* possible additional TLB1 translations are in place (above) so that
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
* all range up to the currently calculated 'data_end' is covered.
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
*/
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
dpcpu_init(dpcpu, 0);
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
memset((void *)ptbl_bufs, 0, sizeof(struct ptbl_buf) * PTBL_SIZE);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
memset((void *)kernel_pdir, 0,
kernel_pdirs * PDIR_PAGES * PAGE_SIZE +
kernel_ptbls * PTBL_PAGES * PAGE_SIZE);
#else
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
memset((void *)kernel_pdir, 0, kernel_ptbls * PTBL_PAGES * PAGE_SIZE);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*******************************************************/
/* Set the start and end of kva. */
/*******************************************************/
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
virtual_avail = round_page(data_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
virtual_end = VM_MAX_KERNEL_ADDRESS;
/* Allocate KVA space for page zero/copy operations. */
zero_page_va = virtual_avail;
virtual_avail += PAGE_SIZE;
copy_page_src_va = virtual_avail;
virtual_avail += PAGE_SIZE;
copy_page_dst_va = virtual_avail;
virtual_avail += PAGE_SIZE;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
debugf("zero_page_va = 0x%08x\n", zero_page_va);
debugf("copy_page_src_va = 0x%08x\n", copy_page_src_va);
debugf("copy_page_dst_va = 0x%08x\n", copy_page_dst_va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Initialize page zero/copy mutexes. */
mtx_init(&zero_page_mutex, "mmu_booke_zero_page", NULL, MTX_DEF);
mtx_init(&copy_page_mutex, "mmu_booke_copy_page", NULL, MTX_DEF);
/* Allocate KVA space for ptbl bufs. */
ptbl_buf_pool_vabase = virtual_avail;
virtual_avail += PTBL_BUFS * PTBL_PAGES * PAGE_SIZE;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
debugf("ptbl_buf_pool_vabase = 0x%08x end = 0x%08x\n",
ptbl_buf_pool_vabase, virtual_avail);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Calculate corresponding physical addresses for the kernel region. */
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
phys_kernelend = kernload + kernsize;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
debugf("kernel image and allocated data:\n");
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
debugf(" kernload = 0x%09llx\n", (uint64_t)kernload);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
debugf(" kernstart = 0x%08x\n", kernstart);
debugf(" kernsize = 0x%08x\n", kernsize);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (sizeof(phys_avail) / sizeof(phys_avail[0]) < availmem_regions_sz)
panic("mmu_booke_bootstrap: phys_avail too small");
/*
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* Remove kernel physical address range from avail regions list. Page
* align all regions. Non-page aligned memory isn't very interesting
* to us. Also, sort the entries for ascending addresses.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
Factor out platform dependent things unrelated to device drivers into a new platform module. These are probed in early boot, and have the responsibility of determining the layout of physical memory, determining the CPU timebase frequency, and handling the zoo of SMP mechanisms found on PowerPC. Reviewed by: marcel, raj Book-E parts by: raj
2009-05-14 00:34:26 +00:00
/* Retrieve phys/avail mem regions */
mem_regions(&physmem_regions, &physmem_regions_sz,
&availmem_regions, &availmem_regions_sz);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
sz = 0;
cnt = availmem_regions_sz;
debugf("processing avail regions:\n");
for (mp = availmem_regions; mp->mr_size; mp++) {
s = mp->mr_start;
e = mp->mr_start + mp->mr_size;
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
debugf(" %09jx-%09jx -> ", (uintmax_t)s, (uintmax_t)e);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Check whether this region holds all of the kernel. */
if (s < kernload && e > phys_kernelend) {
availmem_regions[cnt].mr_start = phys_kernelend;
availmem_regions[cnt++].mr_size = e - phys_kernelend;
e = kernload;
}
/* Look whether this regions starts within the kernel. */
if (s >= kernload && s < phys_kernelend) {
if (e <= phys_kernelend)
goto empty;
s = phys_kernelend;
}
/* Now look whether this region ends within the kernel. */
if (e > kernload && e <= phys_kernelend) {
if (s >= kernload)
goto empty;
e = kernload;
}
/* Now page align the start and size of the region. */
s = round_page(s);
e = trunc_page(e);
if (e < s)
e = s;
sz = e - s;
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
debugf("%09jx-%09jx = %jx\n",
(uintmax_t)s, (uintmax_t)e, (uintmax_t)sz);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Check whether some memory is left here. */
if (sz == 0) {
empty:
memmove(mp, mp + 1,
(cnt - (mp - availmem_regions)) * sizeof(*mp));
cnt--;
mp--;
continue;
}
/* Do an insertion sort. */
for (mp1 = availmem_regions; mp1 < mp; mp1++)
if (s < mp1->mr_start)
break;
if (mp1 < mp) {
memmove(mp1 + 1, mp1, (char *)mp - (char *)mp1);
mp1->mr_start = s;
mp1->mr_size = sz;
} else {
mp->mr_start = s;
mp->mr_size = sz;
}
}
availmem_regions_sz = cnt;
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
/*******************************************************/
/* Steal physical memory for kernel stack from the end */
/* of the first avail region */
/*******************************************************/
Make kstack_pages a tunable on arm, x86, and powepc. On i386, the initial thread stack is not adjusted by the tunable, the stack is allocated too early to get access to the kernel environment. See TD0_KSTACK_PAGES for the thread0 stack sizing on i386. The tunable was tested on x86 only. From the visual inspection, it seems that it might work on arm and powerpc. The arm USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already incorrect for the threads with non-default kstack size. I only changed the macros to use variable instead of constant, since I cannot test. On arm64, mips and sparc64, some static data structures are sized by KSTACK_PAGES, so the tunable is disabled. Sponsored by: The FreeBSD Foundation MFC after: 2 week
2015-08-10 17:18:21 +00:00
kstack0_sz = kstack_pages * PAGE_SIZE;
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
kstack0_phys = availmem_regions[0].mr_start +
availmem_regions[0].mr_size;
kstack0_phys -= kstack0_sz;
availmem_regions[0].mr_size -= kstack0_sz;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*******************************************************/
/* Fill in phys_avail table, based on availmem_regions */
/*******************************************************/
phys_avail_count = 0;
physsz = 0;
hwphyssz = 0;
TUNABLE_ULONG_FETCH("hw.physmem", (u_long *) &hwphyssz);
debugf("fill in phys_avail:\n");
for (i = 0, j = 0; i < availmem_regions_sz; i++, j += 2) {
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
debugf(" region: 0x%jx - 0x%jx (0x%jx)\n",
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
(uintmax_t)availmem_regions[i].mr_start,
(uintmax_t)availmem_regions[i].mr_start +
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
availmem_regions[i].mr_size,
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
(uintmax_t)availmem_regions[i].mr_size);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
if (hwphyssz != 0 &&
(physsz + availmem_regions[i].mr_size) >= hwphyssz) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
debugf(" hw.physmem adjust\n");
if (physsz < hwphyssz) {
phys_avail[j] = availmem_regions[i].mr_start;
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
phys_avail[j + 1] =
availmem_regions[i].mr_start +
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
hwphyssz - physsz;
physsz = hwphyssz;
phys_avail_count++;
}
break;
}
phys_avail[j] = availmem_regions[i].mr_start;
phys_avail[j + 1] = availmem_regions[i].mr_start +
availmem_regions[i].mr_size;
phys_avail_count++;
physsz += availmem_regions[i].mr_size;
}
physmem = btoc(physsz);
/* Calculate the last available physical address. */
for (i = 0; phys_avail[i + 2] != 0; i += 2)
;
Maxmem = powerpc_btop(phys_avail[i + 1]);
debugf("Maxmem = 0x%08lx\n", Maxmem);
debugf("phys_avail_count = %d\n", phys_avail_count);
Fix a debug printf(). Somehow this printf() was missed in the conversion of vm_paddr_t to 64-bit, and made it through until now. Sponsored by: Alex Perez/Inertial Computing
2016-01-26 03:52:14 +00:00
debugf("physsz = 0x%09jx physmem = %jd (0x%09jx)\n",
(uintmax_t)physsz, (uintmax_t)physmem, (uintmax_t)physmem);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*******************************************************/
/* Initialize (statically allocated) kernel pmap. */
/*******************************************************/
PMAP_LOCK_INIT(kernel_pmap);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifndef __powerpc64__
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
kptbl_min = VM_MIN_KERNEL_ADDRESS / PDIR_SIZE;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf("kernel_pmap = 0x%"PRI0ptrX"\n", (uintptr_t)kernel_pmap);
kernel_pte_alloc(virtual_avail, kernstart, kernel_pdir);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
for (i = 0; i < MAXCPU; i++) {
kernel_pmap->pm_tid[i] = TID_KERNEL;
/* Initialize each CPU's tidbusy entry 0 with kernel_pmap */
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
tidbusy[i][TID_KERNEL] = kernel_pmap;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
}
Fill PTEs covering kernel code and data. Without this fix pte_vatopa() was not able to retrieve physical address of data structures inside kernel, for example EFAULT was reported while acessing /dev/kmem ('netstat -nr'). Submitted by: Piotr Ziecik Obtained from: Semihalf
2009-06-05 09:09:46 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Mark kernel_pmap active on all CPUs */
Add the powerpc support. Note that there is a dirty hack for calling openpic_write(), but nwhitehorn approved it. Discussed with: nwhitehorn
2011-05-09 16:16:15 +00:00
CPU_FILL(&kernel_pmap->pm_active);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
/*
* Initialize the global pv list lock.
*/
rw_init(&pvh_global_lock, "pmap pv global");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*******************************************************/
/* Final setup */
/*******************************************************/
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
/* Enter kstack0 into kernel map, provide guard page */
kstack0 = virtual_avail + KSTACK_GUARD_PAGES * PAGE_SIZE;
thread0.td_kstack = kstack0;
Make kstack_pages a tunable on arm, x86, and powepc. On i386, the initial thread stack is not adjusted by the tunable, the stack is allocated too early to get access to the kernel environment. See TD0_KSTACK_PAGES for the thread0 stack sizing on i386. The tunable was tested on x86 only. From the visual inspection, it seems that it might work on arm and powerpc. The arm USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already incorrect for the threads with non-default kstack size. I only changed the macros to use variable instead of constant, since I cannot test. On arm64, mips and sparc64, some static data structures are sized by KSTACK_PAGES, so the tunable is disabled. Sponsored by: The FreeBSD Foundation MFC after: 2 week
2015-08-10 17:18:21 +00:00
thread0.td_kstack_pages = kstack_pages;
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
debugf("kstack_sz = 0x%08x\n", kstack0_sz);
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
debugf("kstack0_phys at 0x%09llx - 0x%09llx\n",
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
kstack0_phys, kstack0_phys + kstack0_sz);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf("kstack0 at 0x%"PRI0ptrX" - 0x%"PRI0ptrX"\n",
kstack0, kstack0 + kstack0_sz);
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
virtual_avail += KSTACK_GUARD_PAGES * PAGE_SIZE + kstack0_sz;
Make kstack_pages a tunable on arm, x86, and powepc. On i386, the initial thread stack is not adjusted by the tunable, the stack is allocated too early to get access to the kernel environment. See TD0_KSTACK_PAGES for the thread0 stack sizing on i386. The tunable was tested on x86 only. From the visual inspection, it seems that it might work on arm and powerpc. The arm USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already incorrect for the threads with non-default kstack size. I only changed the macros to use variable instead of constant, since I cannot test. On arm64, mips and sparc64, some static data structures are sized by KSTACK_PAGES, so the tunable is disabled. Sponsored by: The FreeBSD Foundation MFC after: 2 week
2015-08-10 17:18:21 +00:00
for (i = 0; i < kstack_pages; i++) {
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
mmu_booke_kenter(mmu, kstack0, kstack0_phys);
kstack0 += PAGE_SIZE;
kstack0_phys += PAGE_SIZE;
}
Initialize NX stack capabilities and direct map status in pmap like on AIM.
2015-03-01 21:23:23 +00:00
pmap_bootstrapped = 1;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf("virtual_avail = %"PRI0ptrX"\n", virtual_avail);
debugf("virtual_end = %"PRI0ptrX"\n", virtual_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
debugf("mmu_booke_bootstrap: exit\n");
}
Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and building without SMP causes a build failure. MFC after: 1 month
2015-03-28 21:39:42 +00:00
#ifdef SMP
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
void
tlb1_ap_prep(void)
{
tlb_entry_t *e, tmp;
unsigned int i;
/* Prepare TLB1 image for AP processors */
e = __boot_tlb1;
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&tmp, i);
if ((tmp.mas1 & MAS1_VALID) && (tmp.mas2 & _TLB_ENTRY_SHARED))
memcpy(e++, &tmp, sizeof(tmp));
}
}
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
void
pmap_bootstrap_ap(volatile uint32_t *trcp __unused)
{
int i;
/*
* Finish TLB1 configuration: the BSP already set up its TLB1 and we
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
* have the snapshot of its contents in the s/w __boot_tlb1[] table
* created by tlb1_ap_prep(), so use these values directly to
* (re)program AP's TLB1 hardware.
*
* Start at index 1 because index 0 has the kernel map.
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 1; i < TLB1_ENTRIES; i++) {
if (__boot_tlb1[i].mas1 & MAS1_VALID)
tlb1_write_entry(&__boot_tlb1[i], i);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
}
set_mas4_defaults();
}
Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and building without SMP causes a build failure. MFC after: 1 month
2015-03-28 21:39:42 +00:00
#endif
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
static void
booke_pmap_init_qpages(void)
{
struct pcpu *pc;
int i;
CPU_FOREACH(i) {
pc = pcpu_find(i);
pc->pc_qmap_addr = kva_alloc(PAGE_SIZE);
if (pc->pc_qmap_addr == 0)
panic("pmap_init_qpages: unable to allocate KVA");
}
}
SYSINIT(qpages_init, SI_SUB_CPU, SI_ORDER_ANY, booke_pmap_init_qpages, NULL);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Get the physical page address for the given pmap/virtual address.
*/
static vm_paddr_t
mmu_booke_extract(mmu_t mmu, pmap_t pmap, vm_offset_t va)
{
vm_paddr_t pa;
PMAP_LOCK(pmap);
pa = pte_vatopa(mmu, pmap, va);
PMAP_UNLOCK(pmap);
return (pa);
}
/*
* Extract the physical page address associated with the given
* kernel virtual address.
*/
static vm_paddr_t
mmu_booke_kextract(mmu_t mmu, vm_offset_t va)
{
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Check the page table before TLB1 in pmap_kextract() The vast majority of pmap_kextract() calls are looking for a physical memory address, not a device address. By checking the page table first this saves the formerly inevitable 64 (on e500mc and derivatives) iteration loop through TLB1 in the most common cases. Benchmarking this on the P5020 (e5500 core) yields a 300% throughput improvement on dtsec(4) (115Mbit/s -> 460Mbit/s) measured with iperf. Benchmarked on the P1022 (e500v2 core, 16 TLB1 entries) yields a 50% throughput improvement on tsec(4) (~93Mbit/s -> 165Mbit/s) measured with iperf. MFC after: 1 week Relnotes: Maybe (significant performance improvement)
2017-11-21 03:12:16 +00:00
vm_paddr_t p;
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
int i;
Check the page table before TLB1 in pmap_kextract() The vast majority of pmap_kextract() calls are looking for a physical memory address, not a device address. By checking the page table first this saves the formerly inevitable 64 (on e500mc and derivatives) iteration loop through TLB1 in the most common cases. Benchmarking this on the P5020 (e5500 core) yields a 300% throughput improvement on dtsec(4) (115Mbit/s -> 460Mbit/s) measured with iperf. Benchmarked on the P1022 (e500v2 core, 16 TLB1 entries) yields a 50% throughput improvement on tsec(4) (~93Mbit/s -> 165Mbit/s) measured with iperf. MFC after: 1 week Relnotes: Maybe (significant performance improvement)
2017-11-21 03:12:16 +00:00
p = pte_vatopa(mmu, kernel_pmap, va);
if (p == 0) {
/* Check TLB1 mappings */
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
continue;
if (va >= e.virt && va < e.virt + e.size)
return (e.phys + (va - e.virt));
}
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Check the page table before TLB1 in pmap_kextract() The vast majority of pmap_kextract() calls are looking for a physical memory address, not a device address. By checking the page table first this saves the formerly inevitable 64 (on e500mc and derivatives) iteration loop through TLB1 in the most common cases. Benchmarking this on the P5020 (e5500 core) yields a 300% throughput improvement on dtsec(4) (115Mbit/s -> 460Mbit/s) measured with iperf. Benchmarked on the P1022 (e500v2 core, 16 TLB1 entries) yields a 50% throughput improvement on tsec(4) (~93Mbit/s -> 165Mbit/s) measured with iperf. MFC after: 1 week Relnotes: Maybe (significant performance improvement)
2017-11-21 03:12:16 +00:00
return (p);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Initialize the pmap module.
* Called by vm_init, to initialize any structures that the pmap
* system needs to map virtual memory.
*/
static void
mmu_booke_init(mmu_t mmu)
{
int shpgperproc = PMAP_SHPGPERPROC;
/*
* Initialize the address space (zone) for the pv entries. Set a
* high water mark so that the system can recover from excessive
* numbers of pv entries.
*/
pvzone = uma_zcreate("PV ENTRY", sizeof(struct pv_entry), NULL, NULL,
NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_VM | UMA_ZONE_NOFREE);
TUNABLE_INT_FETCH("vm.pmap.shpgperproc", &shpgperproc);
Rename global cnt to vm_cnt to avoid shadowing. To reduce the diff struct pcu.cnt field was not renamed, so PCPU_OP(cnt.field) is still used. pc_cnt and pcpu are also used in kvm(3) and vmstat(8). The goal was to not affect externally used KPI. Bump __FreeBSD_version_ in case some out-of-tree module/code relies on the the global cnt variable. Exp-run revealed no ports using it directly. No objection from: arch@ Sponsored by: EMC / Isilon Storage Division
2014-03-22 10:26:09 +00:00
pv_entry_max = shpgperproc * maxproc + vm_cnt.v_page_count;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TUNABLE_INT_FETCH("vm.pmap.pv_entries", &pv_entry_max);
pv_entry_high_water = 9 * (pv_entry_max / 10);
Merge from vmc-playground branch: Replace the sub-optimal uma_zone_set_obj() primitive with more modern uma_zone_reserve_kva(). The new primitive reserves before hand the necessary KVA space to cater the zone allocations and allocates pages with ALLOC_NOOBJ. More specifically: - uma_zone_reserve_kva() does not need an object to cater the backend allocator. - uma_zone_reserve_kva() can cater M_WAITOK requests, in order to serve zones which need to do uma_prealloc() too. - When possible, uma_zone_reserve_kva() uses directly the direct-mapping by uma_small_alloc() rather than relying on the KVA / offset combination. The removal of the object attribute allows 2 further changes: 1) _vm_object_allocate() becomes static within vm_object.c 2) VM_OBJECT_LOCK_INIT() is removed. This function is replaced by direct calls to mtx_init() as there is no need to export it anymore and the calls aren't either homogeneous anymore: there are now small differences between arguments passed to mtx_init(). Sponsored by: EMC / Isilon storage division Reviewed by: alc (which also offered almost all the comments) Tested by: pho, jhb, davide
2013-02-26 23:35:27 +00:00
uma_zone_reserve_kva(pvzone, pv_entry_max);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Pre-fill pvzone with initial number of pv entries. */
uma_prealloc(pvzone, PV_ENTRY_ZONE_MIN);
/* Initialize ptbl allocation. */
ptbl_init();
}
/*
* Map a list of wired pages into kernel virtual address space. This is
* intended for temporary mappings which do not need page modification or
* references recorded. Existing mappings in the region are overwritten.
*/
static void
mmu_booke_qenter(mmu_t mmu, vm_offset_t sva, vm_page_t *m, int count)
{
vm_offset_t va;
va = sva;
while (count-- > 0) {
mmu_booke_kenter(mmu, va, VM_PAGE_TO_PHYS(*m));
va += PAGE_SIZE;
m++;
}
}
/*
* Remove page mappings from kernel virtual address space. Intended for
* temporary mappings entered by mmu_booke_qenter.
*/
static void
mmu_booke_qremove(mmu_t mmu, vm_offset_t sva, int count)
{
vm_offset_t va;
va = sva;
while (count-- > 0) {
mmu_booke_kremove(mmu, va);
va += PAGE_SIZE;
}
}
/*
* Map a wired page into kernel virtual address space.
*/
static void
Fix physical address type to vm_paddr_t.
2012-05-24 21:13:24 +00:00
mmu_booke_kenter(mmu_t mmu, vm_offset_t va, vm_paddr_t pa)
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
{
mmu_booke_kenter_attr(mmu, va, pa, VM_MEMATTR_DEFAULT);
}
static void
mmu_booke_kenter_attr(mmu_t mmu, vm_offset_t va, vm_paddr_t pa, vm_memattr_t ma)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint32_t flags;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pte_t *pte;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
KASSERT(((va >= VM_MIN_KERNEL_ADDRESS) &&
(va <= VM_MAX_KERNEL_ADDRESS)), ("mmu_booke_kenter: invalid va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
flags = PTE_SR | PTE_SW | PTE_SX | PTE_WIRED | PTE_VALID;
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
flags |= tlb_calc_wimg(pa, ma) << PTE_MAS2_SHIFT;
flags |= PTE_PS_4KB;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
pte = pte_find(mmu, kernel_pmap, va);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
KASSERT((pte != NULL), ("mmu_booke_kenter: invalid va. NULL PTE"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (PTE_ISVALID(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR1(KTR_PMAP, "%s: replacing entry!", __func__);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Flush entry from TLB0 */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte = PTE_RPN_FROM_PA(pa) | flags;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
//debugf("mmu_booke_kenter: pdir_idx = %d ptbl_idx = %d va=0x%08x "
// "pa=0x%08x rpn=0x%08x flags=0x%08x\n",
// pdir_idx, ptbl_idx, va, pa, pte->rpn, pte->flags);
/* Flush the real memory from the instruction cache. */
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
if ((flags & (PTE_I | PTE_G)) == 0)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
__syncicache((void *)va, PAGE_SIZE);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Remove a page from kernel page table.
*/
static void
mmu_booke_kremove(mmu_t mmu, vm_offset_t va)
{
pte_t *pte;
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
CTR2(KTR_PMAP,"%s: s (va = 0x%08x)\n", __func__, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
KASSERT(((va >= VM_MIN_KERNEL_ADDRESS) &&
(va <= VM_MAX_KERNEL_ADDRESS)),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
("mmu_booke_kremove: invalid va"));
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
pte = pte_find(mmu, kernel_pmap, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!PTE_ISVALID(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR1(KTR_PMAP, "%s: invalid pte", __func__);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return;
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Invalidate entry in TLB0, update PTE. */
tlb0_flush_entry(va);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte = 0;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Initialize pmap associated with process 0.
*/
static void
mmu_booke_pinit0(mmu_t mmu, pmap_t pmap)
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
Revert r254501. Instead, reuse the type stability of the struct pmap which is the part of struct vmspace, allocated from UMA_ZONE_NOFREE zone. Initialize the pmap lock in the vmspace zone init function, and remove pmap lock initialization and destruction from pmap_pinit() and pmap_release(). Suggested and reviewed by: alc (previous version) Tested by: pho Sponsored by: The FreeBSD Foundation
2013-08-22 18:12:24 +00:00
PMAP_LOCK_INIT(pmap);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mmu_booke_pinit(mmu, pmap);
PCPU_SET(curpmap, pmap);
}
/*
* Initialize a preallocated and zeroed pmap structure,
* such as one in a vmspace structure.
*/
static void
mmu_booke_pinit(mmu_t mmu, pmap_t pmap)
{
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
int i;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR4(KTR_PMAP, "%s: pmap = %p, proc %d '%s'", __func__, pmap,
curthread->td_proc->p_pid, curthread->td_proc->p_comm);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
KASSERT((pmap != kernel_pmap), ("pmap_pinit: initializing kernel_pmap"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
for (i = 0; i < MAXCPU; i++)
pmap->pm_tid[i] = TID_NONE;
Add the powerpc support. Note that there is a dirty hack for calling openpic_write(), but nwhitehorn approved it. Discussed with: nwhitehorn
2011-05-09 16:16:15 +00:00
CPU_ZERO(&kernel_pmap->pm_active);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
bzero(&pmap->pm_stats, sizeof(pmap->pm_stats));
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
bzero(&pmap->pm_pp2d, sizeof(pte_t **) * PP2D_NENTRIES);
TAILQ_INIT(&pmap->pm_pdir_list);
#else
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
bzero(&pmap->pm_pdir, sizeof(pte_t *) * PDIR_NENTRIES);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
TAILQ_INIT(&pmap->pm_ptbl_list);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Release any resources held by the given physical map.
* Called when a pmap initialized by mmu_booke_pinit is being released.
* Should only be called if the map contains no valid mappings.
*/
static void
mmu_booke_release(mmu_t mmu, pmap_t pmap)
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
KASSERT(pmap->pm_stats.resident_count == 0,
("pmap_release: pmap resident count %ld != 0",
pmap->pm_stats.resident_count));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Insert the given physical page at the specified virtual address in the
* target physical map with the protection requested. If specified the page
* will be wired down.
*/
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
static int
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mmu_booke_enter(mmu_t mmu, pmap_t pmap, vm_offset_t va, vm_page_t m,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
vm_prot_t prot, u_int flags, int8_t psind)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
int error;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
error = mmu_booke_enter_locked(mmu, pmap, va, m, prot, flags, psind);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pmap);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
rw_wunlock(&pvh_global_lock);
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
return (error);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
static int
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mmu_booke_enter_locked(mmu_t mmu, pmap_t pmap, vm_offset_t va, vm_page_t m,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
vm_prot_t prot, u_int pmap_flags, int8_t psind __unused)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pte_t *pte;
vm_paddr_t pa;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint32_t flags;
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
int error, su, sync;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pa = VM_PAGE_TO_PHYS(m);
su = (pmap == kernel_pmap);
sync = 0;
//debugf("mmu_booke_enter_locked: s (pmap=0x%08x su=%d tid=%d m=0x%08x va=0x%08x "
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
// "pa=0x%08x prot=0x%08x flags=%#x)\n",
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
// (u_int32_t)pmap, su, pmap->pm_tid,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
// (u_int32_t)m, va, pa, prot, flags);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (su) {
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
KASSERT(((va >= virtual_avail) &&
(va <= VM_MAX_KERNEL_ADDRESS)),
("mmu_booke_enter_locked: kernel pmap, non kernel va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
} else {
KASSERT((va <= VM_MAXUSER_ADDRESS),
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
("mmu_booke_enter_locked: user pmap, non user va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
The soft and hard busy mechanism rely on the vm object lock to work. Unify the 2 concept into a real, minimal, sxlock where the shared acquisition represent the soft busy and the exclusive acquisition represent the hard busy. The old VPO_WANTED mechanism becames the hard-path for this new lock and it becomes per-page rather than per-object. The vm_object lock becames an interlock for this functionality: it can be held in both read or write mode. However, if the vm_object lock is held in read mode while acquiring or releasing the busy state, the thread owner cannot make any assumption on the busy state unless it is also busying it. Also: - Add a new flag to directly shared busy pages while vm_page_alloc and vm_page_grab are being executed. This will be very helpful once these functions happen under a read object lock. - Move the swapping sleep into its own per-object flag The KPI is heavilly changed this is why the version is bumped. It is very likely that some VM ports users will need to change their own code. Sponsored by: EMC / Isilon storage division Discussed with: alc Reviewed by: jeff, kib Tested by: gavin, bapt (older version) Tested by: pho, scottl
2013-08-09 11:11:11 +00:00
if ((m->oflags & VPO_UNMANAGED) == 0 && !vm_page_xbusied(m))
Relax the object locking assertion in pmap_enter_locked(). Reviewed by: attilio Sponsored by: EMC / Isilon Storage Division
2013-05-17 18:59:00 +00:00
VM_OBJECT_ASSERT_LOCKED(m->object);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK_ASSERT(pmap, MA_OWNED);
/*
* If there is an existing mapping, and the physical address has not
* changed, must be protection or wiring change.
*/
if (((pte = pte_find(mmu, pmap, va)) != NULL) &&
(PTE_ISVALID(pte)) && (PTE_PA(pte) == pa)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* Before actually updating pte->flags we calculate and
* prepare its new value in a helper var.
*/
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
flags = *pte;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags &= ~(PTE_UW | PTE_UX | PTE_SW | PTE_SX | PTE_MODIFIED);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Wiring change, just update stats. */
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
if ((pmap_flags & PMAP_ENTER_WIRED) != 0) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!PTE_ISWIRED(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_WIRED;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pmap->pm_stats.wired_count++;
}
} else {
if (PTE_ISWIRED(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags &= ~PTE_WIRED;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pmap->pm_stats.wired_count--;
}
}
if (prot & VM_PROT_WRITE) {
/* Add write permissions. */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_SW;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!su)
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_UW;
Set PG_WRITEABLE in Book-E pmap_enter[_locked] if it creates a mapping that permits write access. This is similar to r192671. Pointed out and reviewed by: alc
2009-05-26 06:24:50 +00:00
Don't set PG_WRITEABLE in pmap_enter() unless the page is managed. Correct a typo in a nearby comment on sparc64.
2010-06-05 18:20:09 +00:00
if ((flags & PTE_MANAGED) != 0)
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_set(m, PGA_WRITEABLE);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
} else {
/* Handle modified pages, sense modify status. */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* The PTE_MODIFIED flag could be set by underlying
* TLB misses since we last read it (above), possibly
* other CPUs could update it so we check in the PTE
* directly rather than rely on that saved local flags
* copy.
*/
Eliminate track_modified_needed(), better known as pmap_track_modified() on other platforms. We no longer need it because we do not create managed mappings within the clean submap. Pointed out by: alc
2008-04-27 21:04:54 +00:00
if (PTE_ISMODIFIED(pte))
vm_page_dirty(m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
if (prot & VM_PROT_EXECUTE) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_SX;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!su)
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_UX;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* Check existing flags for execute permissions: if we
* are turning execute permissions on, icache should
* be flushed.
*/
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
if ((*pte & (PTE_UX | PTE_SX)) == 0)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
sync++;
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags &= ~PTE_REFERENCED;
/*
* The new flags value is all calculated -- only now actually
* update the PTE.
*/
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(va);
Fix a panic bug that cropped up in the PTE rewrite. PTE was getting overwritten by just the flags. Pointy-hat to: jhibbits
2016-02-16 02:13:55 +00:00
*pte &= ~PTE_FLAGS_MASK;
*pte |= flags;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
} else {
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* If there is an existing mapping, but it's for a different
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* physical address, pte_enter() will delete the old mapping.
*/
//if ((pte != NULL) && PTE_ISVALID(pte))
// debugf("mmu_booke_enter_locked: replace\n");
//else
// debugf("mmu_booke_enter_locked: new\n");
/* Now set up the flags and install the new mapping. */
flags = (PTE_SR | PTE_VALID);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_M;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!su)
flags |= PTE_UR;
if (prot & VM_PROT_WRITE) {
flags |= PTE_SW;
if (!su)
flags |= PTE_UW;
Set PG_WRITEABLE in Book-E pmap_enter[_locked] if it creates a mapping that permits write access. This is similar to r192671. Pointed out and reviewed by: alc
2009-05-26 06:24:50 +00:00
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
if ((m->oflags & VPO_UNMANAGED) == 0)
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_set(m, PGA_WRITEABLE);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
if (prot & VM_PROT_EXECUTE) {
flags |= PTE_SX;
if (!su)
flags |= PTE_UX;
}
/* If its wired update stats. */
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
if ((pmap_flags & PMAP_ENTER_WIRED) != 0)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
flags |= PTE_WIRED;
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
error = pte_enter(mmu, pmap, m, va, flags,
(pmap_flags & PMAP_ENTER_NOSLEEP) != 0);
if (error != 0)
return (KERN_RESOURCE_SHORTAGE);
if ((flags & PMAP_ENTER_WIRED) != 0)
pmap->pm_stats.wired_count++;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Flush the real memory from the instruction cache. */
if (prot & VM_PROT_EXECUTE)
sync++;
}
if (sync && (su || pmap == PCPU_GET(curpmap))) {
__syncicache((void *)va, PAGE_SIZE);
sync = 0;
}
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
return (KERN_SUCCESS);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Maps a sequence of resident pages belonging to the same object.
* The sequence begins with the given page m_start. This page is
* mapped at the given virtual address start. Each subsequent page is
* mapped at a virtual address that is offset from start by the same
* amount as the page is offset from m_start within the object. The
* last page in the sequence is the page with the largest offset from
* m_start that can be mapped at a virtual address less than the given
* virtual address end. Not every virtual page between start and end
* is mapped; only those for which a resident page exists with the
* corresponding offset from m_start are mapped.
*/
static void
mmu_booke_enter_object(mmu_t mmu, pmap_t pmap, vm_offset_t start,
vm_offset_t end, vm_page_t m_start, vm_prot_t prot)
{
vm_page_t m;
vm_pindex_t diff, psize;
o Relax locking assertions for vm_page_find_least() o Relax locking assertions for pmap_enter_object() and add them also to architectures that currently don't have any o Introduce VM_OBJECT_LOCK_DOWNGRADE() which is basically a downgrade operation on the per-object rwlock o Use all the mechanisms above to make vm_map_pmap_enter() to work mostl of the times only with readlocks. Sponsored by: EMC / Isilon storage division Reviewed by: alc
2013-05-21 20:38:19 +00:00
VM_OBJECT_ASSERT_LOCKED(m_start->object);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
psize = atop(end - start);
m = m_start;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
while (m != NULL && (diff = m->pindex - m_start->pindex) < psize) {
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
mmu_booke_enter_locked(mmu, pmap, start + ptoa(diff), m,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
prot & (VM_PROT_READ | VM_PROT_EXECUTE),
PMAP_ENTER_NOSLEEP, 0);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
m = TAILQ_NEXT(m, listq);
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pmap);
}
static void
mmu_booke_enter_quick(mmu_t mmu, pmap_t pmap, vm_offset_t va, vm_page_t m,
vm_prot_t prot)
{
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
mmu_booke_enter_locked(mmu, pmap, va, m,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
prot & (VM_PROT_READ | VM_PROT_EXECUTE), PMAP_ENTER_NOSLEEP,
0);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pmap);
}
/*
* Remove the given range of addresses from the specified map.
*
* It is assumed that the start and end are properly rounded to the page size.
*/
static void
mmu_booke_remove(mmu_t mmu, pmap_t pmap, vm_offset_t va, vm_offset_t endva)
{
pte_t *pte;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint8_t hold_flag;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
int su = (pmap == kernel_pmap);
//debugf("mmu_booke_remove: s (su = %d pmap=0x%08x tid=%d va=0x%08x endva=0x%08x)\n",
// su, (u_int32_t)pmap, pmap->pm_tid, va, endva);
if (su) {
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
KASSERT(((va >= virtual_avail) &&
(va <= VM_MAX_KERNEL_ADDRESS)),
("mmu_booke_remove: kernel pmap, non kernel va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
} else {
KASSERT((va <= VM_MAXUSER_ADDRESS),
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
("mmu_booke_remove: user pmap, non user va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
if (PMAP_REMOVE_DONE(pmap)) {
//debugf("mmu_booke_remove: e (empty)\n");
return;
}
hold_flag = PTBL_HOLD_FLAG(pmap);
//debugf("mmu_booke_remove: hold_flag = %d\n", hold_flag);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
for (; va < endva; va += PAGE_SIZE) {
pte = pte_find(mmu, pmap, va);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
if ((pte != NULL) && PTE_ISVALID(pte))
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pte_remove(mmu, pmap, va, hold_flag);
}
PMAP_UNLOCK(pmap);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
//debugf("mmu_booke_remove: e\n");
}
/*
* Remove physical page from all pmaps in which it resides.
*/
static void
mmu_booke_remove_all(mmu_t mmu, vm_page_t m)
{
pv_entry_t pv, pvn;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint8_t hold_flag;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
for (pv = TAILQ_FIRST(&m->md.pv_list); pv != NULL; pv = pvn) {
pvn = TAILQ_NEXT(pv, pv_link);
PMAP_LOCK(pv->pv_pmap);
hold_flag = PTBL_HOLD_FLAG(pv->pv_pmap);
pte_remove(mmu, pv->pv_pmap, pv->pv_va, hold_flag);
PMAP_UNLOCK(pv->pv_pmap);
}
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_clear(m, PGA_WRITEABLE);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Map a range of physical addresses into kernel virtual address space.
*/
static vm_offset_t
Fix physical address type to vm_paddr_t.
2012-05-24 21:13:24 +00:00
mmu_booke_map(mmu_t mmu, vm_offset_t *virt, vm_paddr_t pa_start,
vm_paddr_t pa_end, int prot)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
vm_offset_t sva = *virt;
vm_offset_t va = sva;
//debugf("mmu_booke_map: s (sva = 0x%08x pa_start = 0x%08x pa_end = 0x%08x)\n",
// sva, pa_start, pa_end);
while (pa_start < pa_end) {
mmu_booke_kenter(mmu, va, pa_start);
va += PAGE_SIZE;
pa_start += PAGE_SIZE;
}
*virt = va;
//debugf("mmu_booke_map: e (va = 0x%08x)\n", va);
return (sva);
}
/*
* The pmap must be activated before it's address space can be accessed in any
* way.
*/
static void
mmu_booke_activate(mmu_t mmu, struct thread *td)
{
pmap_t pmap;
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
u_int cpuid;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pmap = &td->td_proc->p_vmspace->vm_pmap;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR5(KTR_PMAP, "%s: s (td = %p, proc = '%s', id = %d, pmap = 0x%08x)",
__func__, td, td->td_proc->p_comm, td->td_proc->p_pid, pmap);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((pmap != kernel_pmap), ("mmu_booke_activate: kernel_pmap!"));
Follow up r223485, which made AIM use the ABI thread pointer instead of PCPU fields for curthread, by doing the same to Book-E. This closes some potential races switching between CPUs. As a side effect, it turns out the AIM and Book-E swtch.S implementations were the same to within a few registers, so move that to powerpc/powerpc. MFC after: 3 months
2013-11-11 17:37:50 +00:00
sched_pin();
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
cpuid = PCPU_GET(cpuid);
CPU_SET_ATOMIC(cpuid, &pmap->pm_active);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PCPU_SET(curpmap, pmap);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
if (pmap->pm_tid[cpuid] == TID_NONE)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
tid_alloc(pmap);
/* Load PID0 register with pmap tid value. */
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
mtspr(SPR_PID0, pmap->pm_tid[cpuid]);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
__asm __volatile("isync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Move Book-E/AIM dependent bits for setting user PMAP during thread switch out of cpu_switch() and into pmap_activate() where they belong. This also removes all the #ifdef from cpu_switch().
2015-03-04 16:45:31 +00:00
mtspr(SPR_DBCR0, td->td_pcb->pcb_cpu.booke.dbcr0);
Follow up r223485, which made AIM use the ABI thread pointer instead of PCPU fields for curthread, by doing the same to Book-E. This closes some potential races switching between CPUs. As a side effect, it turns out the AIM and Book-E swtch.S implementations were the same to within a few registers, so move that to powerpc/powerpc. MFC after: 3 months
2013-11-11 17:37:50 +00:00
sched_unpin();
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR3(KTR_PMAP, "%s: e (tid = %d for '%s')", __func__,
pmap->pm_tid[PCPU_GET(cpuid)], td->td_proc->p_comm);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Deactivate the specified process's address space.
*/
static void
mmu_booke_deactivate(mmu_t mmu, struct thread *td)
{
pmap_t pmap;
pmap = &td->td_proc->p_vmspace->vm_pmap;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR5(KTR_PMAP, "%s: td=%p, proc = '%s', id = %d, pmap = 0x%08x",
__func__, td, td->td_proc->p_comm, td->td_proc->p_pid, pmap);
Move Book-E/AIM dependent bits for setting user PMAP during thread switch out of cpu_switch() and into pmap_activate() where they belong. This also removes all the #ifdef from cpu_switch().
2015-03-04 16:45:31 +00:00
td->td_pcb->pcb_cpu.booke.dbcr0 = mfspr(SPR_DBCR0);
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
CPU_CLR_ATOMIC(PCPU_GET(cpuid), &pmap->pm_active);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PCPU_SET(curpmap, NULL);
}
/*
* Copy the range specified by src_addr/len
* from the source map to the range dst_addr/len
* in the destination map.
*
* This routine is only advisory and need not do anything.
*/
static void
Fix Book-E/MPC85XX build. Some prototypes were wrong and got revealed with the recent kobj signature checking.
2009-06-13 08:57:04 +00:00
mmu_booke_copy(mmu_t mmu, pmap_t dst_pmap, pmap_t src_pmap,
vm_offset_t dst_addr, vm_size_t len, vm_offset_t src_addr)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
}
/*
* Set the physical protection on the specified range of this map as requested.
*/
static void
mmu_booke_protect(mmu_t mmu, pmap_t pmap, vm_offset_t sva, vm_offset_t eva,
vm_prot_t prot)
{
vm_offset_t va;
vm_page_t m;
pte_t *pte;
if ((prot & VM_PROT_READ) == VM_PROT_NONE) {
mmu_booke_remove(mmu, pmap, sva, eva);
return;
}
if (prot & VM_PROT_WRITE)
return;
PMAP_LOCK(pmap);
for (va = sva; va < eva; va += PAGE_SIZE) {
if ((pte = pte_find(mmu, pmap, va)) != NULL) {
if (PTE_ISVALID(pte)) {
m = PHYS_TO_VM_PAGE(PTE_PA(pte));
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Handle modified pages. */
MFamd64/i386 r207205 Clearing a page table entry's accessed bit and setting the page's PG_REFERENCED flag in pmap_protect() can't really be justified, so don't do it. Additionally, two changes that make this pmap behave like the others do: Change pmap_protect() such that it calls vm_page_dirty() only if the page is managed. Change pmap_remove_write() such that it doesn't clear a page table entry's accessed bit.
2010-04-30 15:22:52 +00:00
if (PTE_ISMODIFIED(pte) && PTE_ISMANAGED(pte))
Eliminate track_modified_needed(), better known as pmap_track_modified() on other platforms. We no longer need it because we do not create managed mappings within the clean submap. Pointed out by: alc
2008-04-27 21:04:54 +00:00
vm_page_dirty(m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(va);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~(PTE_UW | PTE_SW | PTE_MODIFIED);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
}
}
PMAP_UNLOCK(pmap);
}
/*
* Clear the write and modified bits in each of the given page's mappings.
*/
static void
mmu_booke_remove_write(mmu_t mmu, vm_page_t m)
{
pv_entry_t pv;
pte_t *pte;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
On entry to pmap_enter(), assert that the page is busy. While I'm here, make the style of assertion used by pmap_enter() consistent across all architectures. On entry to pmap_remove_write(), assert that the page is neither unmanaged nor fictitious, since we cannot remove write access to either kind of page. With the push down of the page queues lock, pmap_remove_write() cannot condition its behavior on the state of the PG_WRITEABLE flag if the page is busy. Assert that the object containing the page is locked. This allows us to know that the page will neither become busy nor will PG_WRITEABLE be set on it while pmap_remove_write() is running. Correct a long-standing bug in vm_page_cowsetup(). We cannot possibly do copy-on-write-based zero-copy transmit on unmanaged or fictitious pages, so don't even try. Previously, the call to pmap_remove_write() would have failed silently.
2010-05-16 23:45:10 +00:00
("mmu_booke_remove_write: page %p is not managed", m));
/*
The soft and hard busy mechanism rely on the vm object lock to work. Unify the 2 concept into a real, minimal, sxlock where the shared acquisition represent the soft busy and the exclusive acquisition represent the hard busy. The old VPO_WANTED mechanism becames the hard-path for this new lock and it becomes per-page rather than per-object. The vm_object lock becames an interlock for this functionality: it can be held in both read or write mode. However, if the vm_object lock is held in read mode while acquiring or releasing the busy state, the thread owner cannot make any assumption on the busy state unless it is also busying it. Also: - Add a new flag to directly shared busy pages while vm_page_alloc and vm_page_grab are being executed. This will be very helpful once these functions happen under a read object lock. - Move the swapping sleep into its own per-object flag The KPI is heavilly changed this is why the version is bumped. It is very likely that some VM ports users will need to change their own code. Sponsored by: EMC / Isilon storage division Discussed with: alc Reviewed by: jeff, kib Tested by: gavin, bapt (older version) Tested by: pho, scottl
2013-08-09 11:11:11 +00:00
* If the page is not exclusive busied, then PGA_WRITEABLE cannot be
* set by another thread while the object is locked. Thus,
* if PGA_WRITEABLE is clear, no page table entries need updating.
On entry to pmap_enter(), assert that the page is busy. While I'm here, make the style of assertion used by pmap_enter() consistent across all architectures. On entry to pmap_remove_write(), assert that the page is neither unmanaged nor fictitious, since we cannot remove write access to either kind of page. With the push down of the page queues lock, pmap_remove_write() cannot condition its behavior on the state of the PG_WRITEABLE flag if the page is busy. Assert that the object containing the page is locked. This allows us to know that the page will neither become busy nor will PG_WRITEABLE be set on it while pmap_remove_write() is running. Correct a long-standing bug in vm_page_cowsetup(). We cannot possibly do copy-on-write-based zero-copy transmit on unmanaged or fictitious pages, so don't even try. Previously, the call to pmap_remove_write() would have failed silently.
2010-05-16 23:45:10 +00:00
*/
Switch the vm_object mutex to be a rwlock. This will enable in the future further optimizations where the vm_object lock will be held in read mode most of the time the page cache resident pool of pages are accessed for reading purposes. The change is mostly mechanical but few notes are reported: * The KPI changes as follow: - VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK() - VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK() - VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK() - VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED() (in order to avoid visibility of implementation details) - The read-mode operations are added: VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(), VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED() * The vm/vm_pager.h namespace pollution avoidance (forcing requiring sys/mutex.h in consumers directly to cater its inlining functions using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h consumers now must include also sys/rwlock.h. * zfs requires a quite convoluted fix to include FreeBSD rwlocks into the compat layer because the name clash between FreeBSD and solaris versions must be avoided. At this purpose zfs redefines the vm_object locking functions directly, isolating the FreeBSD components in specific compat stubs. The KPI results heavilly broken by this commit. Thirdy part ports must be updated accordingly (I can think off-hand of VirtualBox, for example). Sponsored by: EMC / Isilon storage division Reviewed by: jeff Reviewed by: pjd (ZFS specific review) Discussed with: alc Tested by: pho
2013-03-09 02:32:23 +00:00
VM_OBJECT_ASSERT_WLOCKED(m->object);
The soft and hard busy mechanism rely on the vm object lock to work. Unify the 2 concept into a real, minimal, sxlock where the shared acquisition represent the soft busy and the exclusive acquisition represent the hard busy. The old VPO_WANTED mechanism becames the hard-path for this new lock and it becomes per-page rather than per-object. The vm_object lock becames an interlock for this functionality: it can be held in both read or write mode. However, if the vm_object lock is held in read mode while acquiring or releasing the busy state, the thread owner cannot make any assumption on the busy state unless it is also busying it. Also: - Add a new flag to directly shared busy pages while vm_page_alloc and vm_page_grab are being executed. This will be very helpful once these functions happen under a read object lock. - Move the swapping sleep into its own per-object flag The KPI is heavilly changed this is why the version is bumped. It is very likely that some VM ports users will need to change their own code. Sponsored by: EMC / Isilon storage division Discussed with: alc Reviewed by: jeff, kib Tested by: gavin, bapt (older version) Tested by: pho, scottl
2013-08-09 11:11:11 +00:00
if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
if ((pte = pte_find(mmu, pv->pv_pmap, pv->pv_va)) != NULL) {
if (PTE_ISVALID(pte)) {
m = PHYS_TO_VM_PAGE(PTE_PA(pte));
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Handle modified pages. */
Eliminate track_modified_needed(), better known as pmap_track_modified() on other platforms. We no longer need it because we do not create managed mappings within the clean submap. Pointed out by: alc
2008-04-27 21:04:54 +00:00
if (PTE_ISMODIFIED(pte))
vm_page_dirty(m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Flush mapping from TLB0. */
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~(PTE_UW | PTE_SW | PTE_MODIFIED);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
}
PMAP_UNLOCK(pv->pv_pmap);
}
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_clear(m, PGA_WRITEABLE);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked). The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
2009-10-21 18:38:02 +00:00
static void
mmu_booke_sync_icache(mmu_t mmu, pmap_t pm, vm_offset_t va, vm_size_t sz)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pte_t *pte;
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked). The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
2009-10-21 18:38:02 +00:00
pmap_t pmap;
vm_page_t m;
vm_offset_t addr;
Avoid spurious compiler warning about an uninitialized variable.
2014-02-01 20:06:52 +00:00
vm_paddr_t pa = 0;
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked). The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
2009-10-21 18:38:02 +00:00
int active, valid;
va = trunc_page(va);
sz = round_page(sz);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked). The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
2009-10-21 18:38:02 +00:00
pmap = PCPU_GET(curpmap);
active = (pm == kernel_pmap || pm == pmap) ? 1 : 0;
while (sz > 0) {
PMAP_LOCK(pm);
pte = pte_find(mmu, pm, va);
valid = (pte != NULL && PTE_ISVALID(pte)) ? 1 : 0;
if (valid)
pa = PTE_PA(pte);
PMAP_UNLOCK(pm);
if (valid) {
if (!active) {
/* Create a mapping in the active pmap. */
addr = 0;
m = PHYS_TO_VM_PAGE(pa);
PMAP_LOCK(pmap);
pte_enter(mmu, pmap, m, addr,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
PTE_SR | PTE_VALID | PTE_UR, FALSE);
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked). The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
2009-10-21 18:38:02 +00:00
__syncicache((void *)addr, PAGE_SIZE);
pte_remove(mmu, pmap, addr, PTBL_UNHOLD);
PMAP_UNLOCK(pmap);
} else
__syncicache((void *)va, PAGE_SIZE);
}
va += PAGE_SIZE;
sz -= PAGE_SIZE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Atomically extract and hold the physical page with the given
* pmap and virtual address pair if that mapping permits the given
* protection.
*/
static vm_page_t
mmu_booke_extract_and_hold(mmu_t mmu, pmap_t pmap, vm_offset_t va,
vm_prot_t prot)
{
pte_t *pte;
vm_page_t m;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint32_t pte_wbit;
On Alan's advice, rather than do a wholesale conversion on a single architecture from page queue lock to a hashed array of page locks (based on a patch by Jeff Roberson), I've implemented page lock support in the MI code and have only moved vm_page's hold_count out from under page queue mutex to page lock. This changes pmap_extract_and_hold on all pmaps. Supported by: Bitgravity Inc. Discussed with: alc, jeffr, and kib
2010-04-30 00:46:43 +00:00
vm_paddr_t pa;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
m = NULL;
On Alan's advice, rather than do a wholesale conversion on a single architecture from page queue lock to a hashed array of page locks (based on a patch by Jeff Roberson), I've implemented page lock support in the MI code and have only moved vm_page's hold_count out from under page queue mutex to page lock. This changes pmap_extract_and_hold on all pmaps. Supported by: Bitgravity Inc. Discussed with: alc, jeffr, and kib
2010-04-30 00:46:43 +00:00
pa = 0;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
On Alan's advice, rather than do a wholesale conversion on a single architecture from page queue lock to a hashed array of page locks (based on a patch by Jeff Roberson), I've implemented page lock support in the MI code and have only moved vm_page's hold_count out from under page queue mutex to page lock. This changes pmap_extract_and_hold on all pmaps. Supported by: Bitgravity Inc. Discussed with: alc, jeffr, and kib
2010-04-30 00:46:43 +00:00
retry:
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
pte = pte_find(mmu, pmap, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if ((pte != NULL) && PTE_ISVALID(pte)) {
if (pmap == kernel_pmap)
pte_wbit = PTE_SW;
else
pte_wbit = PTE_UW;
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
if ((*pte & pte_wbit) || ((prot & VM_PROT_WRITE) == 0)) {
On Alan's advice, rather than do a wholesale conversion on a single architecture from page queue lock to a hashed array of page locks (based on a patch by Jeff Roberson), I've implemented page lock support in the MI code and have only moved vm_page's hold_count out from under page queue mutex to page lock. This changes pmap_extract_and_hold on all pmaps. Supported by: Bitgravity Inc. Discussed with: alc, jeffr, and kib
2010-04-30 00:46:43 +00:00
if (vm_page_pa_tryrelock(pmap, PTE_PA(pte), &pa))
goto retry;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
m = PHYS_TO_VM_PAGE(PTE_PA(pte));
vm_page_hold(m);
}
}
On Alan's advice, rather than do a wholesale conversion on a single architecture from page queue lock to a hashed array of page locks (based on a patch by Jeff Roberson), I've implemented page lock support in the MI code and have only moved vm_page's hold_count out from under page queue mutex to page lock. This changes pmap_extract_and_hold on all pmaps. Supported by: Bitgravity Inc. Discussed with: alc, jeffr, and kib
2010-04-30 00:46:43 +00:00
PA_UNLOCK_COND(pa);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pmap);
return (m);
}
/*
* Initialize a vm_page's machine-dependent fields.
*/
static void
mmu_booke_page_init(mmu_t mmu, vm_page_t m)
{
Properly initialize the full md_page structure
2017-11-10 04:23:58 +00:00
m->md.pv_tracked = 0;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_INIT(&m->md.pv_list);
}
/*
* mmu_booke_zero_page_area zeros the specified hardware page by
* mapping it into virtual memory and using bzero to clear
* its contents.
*
* off and size must reside within a single page.
*/
static void
mmu_booke_zero_page_area(mmu_t mmu, vm_page_t m, int off, int size)
{
vm_offset_t va;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
/* XXX KASSERT off and size are within a single page? */
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mtx_lock(&zero_page_mutex);
va = zero_page_va;
mmu_booke_kenter(mmu, va, VM_PAGE_TO_PHYS(m));
bzero((caddr_t)va + off, size);
mmu_booke_kremove(mmu, va);
mtx_unlock(&zero_page_mutex);
}
/*
* mmu_booke_zero_page zeros the specified hardware page.
*/
static void
mmu_booke_zero_page(mmu_t mmu, vm_page_t m)
{
Optimize zero_page for book-e mmu. Instead of indirectly calling bzero() through mmu_booke_zero_page_area, zero the full page the same way as the AIM pmap logic does: using dcbz.
2015-12-30 02:26:04 +00:00
vm_offset_t off, va;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Optimize zero_page for book-e mmu. Instead of indirectly calling bzero() through mmu_booke_zero_page_area, zero the full page the same way as the AIM pmap logic does: using dcbz.
2015-12-30 02:26:04 +00:00
mtx_lock(&zero_page_mutex);
va = zero_page_va;
mmu_booke_kenter(mmu, va, VM_PAGE_TO_PHYS(m));
for (off = 0; off < PAGE_SIZE; off += cacheline_size)
Older Book-E processors (e500v1/e500v2) don't support dcbzl. The only difference between dcbzl and dcbz is dcbzl operates on native cache line lengths regardless of L1CSR0[DCBZ32]. Since we don't change the cache line size, the cacheline_size variable will reflect the used cache line length, and dcbz will work as expected.
2016-01-26 04:41:18 +00:00
__asm __volatile("dcbz 0,%0" :: "r"(va + off));
Optimize zero_page for book-e mmu. Instead of indirectly calling bzero() through mmu_booke_zero_page_area, zero the full page the same way as the AIM pmap logic does: using dcbz.
2015-12-30 02:26:04 +00:00
mmu_booke_kremove(mmu, va);
mtx_unlock(&zero_page_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* mmu_booke_copy_page copies the specified (machine independent) page by
* mapping the page into virtual memory and using memcopy to copy the page,
* one machine dependent page at a time.
*/
static void
mmu_booke_copy_page(mmu_t mmu, vm_page_t sm, vm_page_t dm)
{
vm_offset_t sva, dva;
sva = copy_page_src_va;
dva = copy_page_dst_va;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock(&copy_page_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mmu_booke_kenter(mmu, sva, VM_PAGE_TO_PHYS(sm));
mmu_booke_kenter(mmu, dva, VM_PAGE_TO_PHYS(dm));
memcpy((caddr_t)dva, (caddr_t)sva, PAGE_SIZE);
mmu_booke_kremove(mmu, dva);
mmu_booke_kremove(mmu, sva);
mtx_unlock(&copy_page_mutex);
}
Add pmap function pmap_copy_pages(), which copies the content of the pages around, taking array of vm_page_t both for source and destination. Starting offsets and total transfer size are specified. The function implements optimal algorithm for copying using the platform-specific optimizations. For instance, on the architectures were the direct map is available, no transient mappings are created, for i386 the per-cpu ephemeral page frame is used. The code was typically borrowed from the pmap_copy_page() for the same architecture. Only i386/amd64, powerpc aim and arm/arm-v6 implementations were tested at the time of commit. High-level code, not committed yet to the tree, ensures that the use of the function is only allowed after explicit enablement. For sparc64, the existing code has known issues and a stab is added instead, to allow the kernel linking. Sponsored by: The FreeBSD Foundation Tested by: pho (i386, amd64), scottl (amd64), ian (arm and arm-v6) MFC after: 2 weeks
2013-03-14 20:18:12 +00:00
static inline void
mmu_booke_copy_pages(mmu_t mmu, vm_page_t *ma, vm_offset_t a_offset,
vm_page_t *mb, vm_offset_t b_offset, int xfersize)
{
void *a_cp, *b_cp;
vm_offset_t a_pg_offset, b_pg_offset;
int cnt;
mtx_lock(&copy_page_mutex);
while (xfersize > 0) {
a_pg_offset = a_offset & PAGE_MASK;
cnt = min(xfersize, PAGE_SIZE - a_pg_offset);
mmu_booke_kenter(mmu, copy_page_src_va,
VM_PAGE_TO_PHYS(ma[a_offset >> PAGE_SHIFT]));
a_cp = (char *)copy_page_src_va + a_pg_offset;
b_pg_offset = b_offset & PAGE_MASK;
cnt = min(cnt, PAGE_SIZE - b_pg_offset);
mmu_booke_kenter(mmu, copy_page_dst_va,
VM_PAGE_TO_PHYS(mb[b_offset >> PAGE_SHIFT]));
b_cp = (char *)copy_page_dst_va + b_pg_offset;
bcopy(a_cp, b_cp, cnt);
mmu_booke_kremove(mmu, copy_page_dst_va);
mmu_booke_kremove(mmu, copy_page_src_va);
a_offset += cnt;
b_offset += cnt;
xfersize -= cnt;
}
mtx_unlock(&copy_page_mutex);
}
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
static vm_offset_t
mmu_booke_quick_enter_page(mmu_t mmu, vm_page_t m)
{
vm_paddr_t paddr;
vm_offset_t qaddr;
uint32_t flags;
pte_t *pte;
paddr = VM_PAGE_TO_PHYS(m);
flags = PTE_SR | PTE_SW | PTE_SX | PTE_WIRED | PTE_VALID;
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
flags |= tlb_calc_wimg(paddr, pmap_page_get_memattr(m)) << PTE_MAS2_SHIFT;
flags |= PTE_PS_4KB;
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
critical_enter();
qaddr = PCPU_GET(qmap_addr);
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
pte = pte_find(mmu, kernel_pmap, qaddr);
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
KASSERT(*pte == 0, ("mmu_booke_quick_enter_page: PTE busy"));
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
/*
* XXX: tlbivax is broadcast to other cores, but qaddr should
* not be present in other TLBs. Is there a better instruction
* sequence to use? Or just forget it & use mmu_booke_kenter()...
*/
__asm __volatile("tlbivax 0, %0" :: "r"(qaddr & MAS2_EPN_MASK));
__asm __volatile("isync; msync");
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte = PTE_RPN_FROM_PA(paddr) | flags;
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
/* Flush the real memory from the instruction cache. */
if ((flags & (PTE_I | PTE_G)) == 0)
__syncicache((void *)qaddr, PAGE_SIZE);
return (qaddr);
}
static void
mmu_booke_quick_remove_page(mmu_t mmu, vm_offset_t addr)
{
pte_t *pte;
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
pte = pte_find(mmu, kernel_pmap, addr);
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
KASSERT(PCPU_GET(qmap_addr) == addr,
("mmu_booke_quick_remove_page: invalid address"));
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
KASSERT(*pte != 0,
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
("mmu_booke_quick_remove_page: PTE not in use"));
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte = 0;
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
critical_exit();
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Return whether or not the specified physical page was modified
* in any of physical maps.
*/
static boolean_t
mmu_booke_is_modified(mmu_t mmu, vm_page_t m)
{
pte_t *pte;
pv_entry_t pv;
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
boolean_t rv;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
("mmu_booke_is_modified: page %p is not managed", m));
rv = FALSE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
/*
The soft and hard busy mechanism rely on the vm object lock to work. Unify the 2 concept into a real, minimal, sxlock where the shared acquisition represent the soft busy and the exclusive acquisition represent the hard busy. The old VPO_WANTED mechanism becames the hard-path for this new lock and it becomes per-page rather than per-object. The vm_object lock becames an interlock for this functionality: it can be held in both read or write mode. However, if the vm_object lock is held in read mode while acquiring or releasing the busy state, the thread owner cannot make any assumption on the busy state unless it is also busying it. Also: - Add a new flag to directly shared busy pages while vm_page_alloc and vm_page_grab are being executed. This will be very helpful once these functions happen under a read object lock. - Move the swapping sleep into its own per-object flag The KPI is heavilly changed this is why the version is bumped. It is very likely that some VM ports users will need to change their own code. Sponsored by: EMC / Isilon storage division Discussed with: alc Reviewed by: jeff, kib Tested by: gavin, bapt (older version) Tested by: pho, scottl
2013-08-09 11:11:11 +00:00
* If the page is not exclusive busied, then PGA_WRITEABLE cannot be
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
* concurrently set while the object is locked. Thus, if PGA_WRITEABLE
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
* is clear, no PTEs can be modified.
*/
Switch the vm_object mutex to be a rwlock. This will enable in the future further optimizations where the vm_object lock will be held in read mode most of the time the page cache resident pool of pages are accessed for reading purposes. The change is mostly mechanical but few notes are reported: * The KPI changes as follow: - VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK() - VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK() - VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK() - VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED() (in order to avoid visibility of implementation details) - The read-mode operations are added: VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(), VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED() * The vm/vm_pager.h namespace pollution avoidance (forcing requiring sys/mutex.h in consumers directly to cater its inlining functions using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h consumers now must include also sys/rwlock.h. * zfs requires a quite convoluted fix to include FreeBSD rwlocks into the compat layer because the name clash between FreeBSD and solaris versions must be avoided. At this purpose zfs redefines the vm_object locking functions directly, isolating the FreeBSD components in specific compat stubs. The KPI results heavilly broken by this commit. Thirdy part ports must be updated accordingly (I can think off-hand of VirtualBox, for example). Sponsored by: EMC / Isilon storage division Reviewed by: jeff Reviewed by: pjd (ZFS specific review) Discussed with: alc Tested by: pho
2013-03-09 02:32:23 +00:00
VM_OBJECT_ASSERT_WLOCKED(m->object);
The soft and hard busy mechanism rely on the vm object lock to work. Unify the 2 concept into a real, minimal, sxlock where the shared acquisition represent the soft busy and the exclusive acquisition represent the hard busy. The old VPO_WANTED mechanism becames the hard-path for this new lock and it becomes per-page rather than per-object. The vm_object lock becames an interlock for this functionality: it can be held in both read or write mode. However, if the vm_object lock is held in read mode while acquiring or releasing the busy state, the thread owner cannot make any assumption on the busy state unless it is also busying it. Also: - Add a new flag to directly shared busy pages while vm_page_alloc and vm_page_grab are being executed. This will be very helpful once these functions happen under a read object lock. - Move the swapping sleep into its own per-object flag The KPI is heavilly changed this is why the version is bumped. It is very likely that some VM ports users will need to change their own code. Sponsored by: EMC / Isilon storage division Discussed with: alc Reviewed by: jeff, kib Tested by: gavin, bapt (older version) Tested by: pho, scottl
2013-08-09 11:11:11 +00:00
if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0)
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
return (rv);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
if ((pte = pte_find(mmu, pv->pv_pmap, pv->pv_va)) != NULL &&
PTE_ISVALID(pte)) {
if (PTE_ISMODIFIED(pte))
rv = TRUE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
PMAP_UNLOCK(pv->pv_pmap);
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
if (rv)
break;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
return (rv);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* Return whether or not the specified virtual address is eligible
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* for prefault.
*/
static boolean_t
mmu_booke_is_prefaultable(mmu_t mmu, pmap_t pmap, vm_offset_t addr)
{
return (FALSE);
}
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
/*
* Return whether or not the specified physical page was referenced
* in any physical maps.
*/
static boolean_t
mmu_booke_is_referenced(mmu_t mmu, vm_page_t m)
{
pte_t *pte;
pv_entry_t pv;
boolean_t rv;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Push down page queues lock acquisition in pmap_enter_object() and pmap_is_referenced(). Eliminate the corresponding page queues lock acquisitions from vm_map_pmap_enter() and mincore(), respectively. In mincore(), this allows some additional cases to complete without ever acquiring the page queues lock. Assert that the page is managed in pmap_is_referenced(). On powerpc/aim, push down the page queues lock acquisition from moea*_is_modified() and moea*_is_referenced() into moea*_query_bit(). Again, this will allow some additional cases to complete without ever acquiring the page queues lock. Reorder a few statements in vm_page_dontneed() so that a race can't lead to an old reference persisting. This scenario is described in detail by a comment. Correct a spelling error in vm_page_dontneed(). Assert that the object is locked in vm_page_clear_dirty(), and restrict the page queues lock assertion to just those cases in which the page is currently writeable. Add object locking to vnode_pager_generic_putpages(). This was the one and only place where vm_page_clear_dirty() was being called without the object being locked. Eliminate an unnecessary vm_page_lock() around vnode_pager_setsize()'s call to vm_page_clear_dirty(). Change vnode_pager_generic_putpages() to the modern-style of function definition. Also, change the name of one of the parameters to follow virtual memory system naming conventions. Reviewed by: kib
2010-05-26 18:00:44 +00:00
("mmu_booke_is_referenced: page %p is not managed", m));
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
rv = FALSE;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
if ((pte = pte_find(mmu, pv->pv_pmap, pv->pv_va)) != NULL &&
Push down page queues lock acquisition in pmap_enter_object() and pmap_is_referenced(). Eliminate the corresponding page queues lock acquisitions from vm_map_pmap_enter() and mincore(), respectively. In mincore(), this allows some additional cases to complete without ever acquiring the page queues lock. Assert that the page is managed in pmap_is_referenced(). On powerpc/aim, push down the page queues lock acquisition from moea*_is_modified() and moea*_is_referenced() into moea*_query_bit(). Again, this will allow some additional cases to complete without ever acquiring the page queues lock. Reorder a few statements in vm_page_dontneed() so that a race can't lead to an old reference persisting. This scenario is described in detail by a comment. Correct a spelling error in vm_page_dontneed(). Assert that the object is locked in vm_page_clear_dirty(), and restrict the page queues lock assertion to just those cases in which the page is currently writeable. Add object locking to vnode_pager_generic_putpages(). This was the one and only place where vm_page_clear_dirty() was being called without the object being locked. Eliminate an unnecessary vm_page_lock() around vnode_pager_setsize()'s call to vm_page_clear_dirty(). Change vnode_pager_generic_putpages() to the modern-style of function definition. Also, change the name of one of the parameters to follow virtual memory system naming conventions. Reviewed by: kib
2010-05-26 18:00:44 +00:00
PTE_ISVALID(pte)) {
if (PTE_ISREFERENCED(pte))
rv = TRUE;
}
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
PMAP_UNLOCK(pv->pv_pmap);
if (rv)
break;
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
return (rv);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Clear the modify bits on the specified physical page.
*/
static void
mmu_booke_clear_modify(mmu_t mmu, vm_page_t m)
{
pte_t *pte;
pv_entry_t pv;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
("mmu_booke_clear_modify: page %p is not managed", m));
Switch the vm_object mutex to be a rwlock. This will enable in the future further optimizations where the vm_object lock will be held in read mode most of the time the page cache resident pool of pages are accessed for reading purposes. The change is mostly mechanical but few notes are reported: * The KPI changes as follow: - VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK() - VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK() - VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK() - VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED() (in order to avoid visibility of implementation details) - The read-mode operations are added: VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(), VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED() * The vm/vm_pager.h namespace pollution avoidance (forcing requiring sys/mutex.h in consumers directly to cater its inlining functions using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h consumers now must include also sys/rwlock.h. * zfs requires a quite convoluted fix to include FreeBSD rwlocks into the compat layer because the name clash between FreeBSD and solaris versions must be avoided. At this purpose zfs redefines the vm_object locking functions directly, isolating the FreeBSD components in specific compat stubs. The KPI results heavilly broken by this commit. Thirdy part ports must be updated accordingly (I can think off-hand of VirtualBox, for example). Sponsored by: EMC / Isilon storage division Reviewed by: jeff Reviewed by: pjd (ZFS specific review) Discussed with: alc Tested by: pho
2013-03-09 02:32:23 +00:00
VM_OBJECT_ASSERT_WLOCKED(m->object);
The soft and hard busy mechanism rely on the vm object lock to work. Unify the 2 concept into a real, minimal, sxlock where the shared acquisition represent the soft busy and the exclusive acquisition represent the hard busy. The old VPO_WANTED mechanism becames the hard-path for this new lock and it becomes per-page rather than per-object. The vm_object lock becames an interlock for this functionality: it can be held in both read or write mode. However, if the vm_object lock is held in read mode while acquiring or releasing the busy state, the thread owner cannot make any assumption on the busy state unless it is also busying it. Also: - Add a new flag to directly shared busy pages while vm_page_alloc and vm_page_grab are being executed. This will be very helpful once these functions happen under a read object lock. - Move the swapping sleep into its own per-object flag The KPI is heavilly changed this is why the version is bumped. It is very likely that some VM ports users will need to change their own code. Sponsored by: EMC / Isilon storage division Discussed with: alc Reviewed by: jeff, kib Tested by: gavin, bapt (older version) Tested by: pho, scottl
2013-08-09 11:11:11 +00:00
KASSERT(!vm_page_xbusied(m),
("mmu_booke_clear_modify: page %p is exclusive busied", m));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
/*
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
* If the page is not PG_AWRITEABLE, then no PTEs can be modified.
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
* If the object containing the page is locked and the page is not
The soft and hard busy mechanism rely on the vm object lock to work. Unify the 2 concept into a real, minimal, sxlock where the shared acquisition represent the soft busy and the exclusive acquisition represent the hard busy. The old VPO_WANTED mechanism becames the hard-path for this new lock and it becomes per-page rather than per-object. The vm_object lock becames an interlock for this functionality: it can be held in both read or write mode. However, if the vm_object lock is held in read mode while acquiring or releasing the busy state, the thread owner cannot make any assumption on the busy state unless it is also busying it. Also: - Add a new flag to directly shared busy pages while vm_page_alloc and vm_page_grab are being executed. This will be very helpful once these functions happen under a read object lock. - Move the swapping sleep into its own per-object flag The KPI is heavilly changed this is why the version is bumped. It is very likely that some VM ports users will need to change their own code. Sponsored by: EMC / Isilon storage division Discussed with: alc Reviewed by: jeff, kib Tested by: gavin, bapt (older version) Tested by: pho, scottl
2013-08-09 11:11:11 +00:00
* exclusive busied, then PG_AWRITEABLE cannot be concurrently set.
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
*/
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
if ((m->aflags & PGA_WRITEABLE) == 0)
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
return;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
if ((pte = pte_find(mmu, pv->pv_pmap, pv->pv_va)) != NULL &&
PTE_ISVALID(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
if (*pte & (PTE_SW | PTE_UW | PTE_MODIFIED)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(pv->pv_va);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~(PTE_SW | PTE_UW | PTE_MODIFIED |
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PTE_REFERENCED);
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
PMAP_UNLOCK(pv->pv_pmap);
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Return a count of reference bits for a page, clearing those bits.
* It is not necessary for every reference bit to be cleared, but it
* is necessary that 0 only be returned when there are truly no
* reference bits set.
*
Various changes to pmap_ts_referenced() Move PMAP_TS_REFERENCED_MAX out of the various pmap implementations and into vm/pmap.h, and describe what its purpose is. Eliminate the archaic "XXX" comment about its value. I don't believe that its exact value, e.g., 5 versus 6, matters. Update the arm64 and riscv pmap implementations of pmap_ts_referenced() to opportunistically update the page's dirty field. On amd64, use the PDE value already cached in a local variable rather than dereferencing a pointer again and again. Reviewed by: kib, markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D7836
2016-09-10 16:49:25 +00:00
* As an optimization, update the page's dirty field if a modified bit is
* found while counting reference bits. This opportunistic update can be
* performed at low cost and can eliminate the need for some future calls
* to pmap_is_modified(). However, since this function stops after
* finding PMAP_TS_REFERENCED_MAX reference bits, it may not detect some
* dirty pages. Those dirty pages will only be detected by a future call
* to pmap_is_modified().
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
static int
mmu_booke_ts_referenced(mmu_t mmu, vm_page_t m)
{
pte_t *pte;
pv_entry_t pv;
int count;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
("mmu_booke_ts_referenced: page %p is not managed", m));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
count = 0;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
if ((pte = pte_find(mmu, pv->pv_pmap, pv->pv_va)) != NULL &&
PTE_ISVALID(pte)) {
Various changes to pmap_ts_referenced() Move PMAP_TS_REFERENCED_MAX out of the various pmap implementations and into vm/pmap.h, and describe what its purpose is. Eliminate the archaic "XXX" comment about its value. I don't believe that its exact value, e.g., 5 versus 6, matters. Update the arm64 and riscv pmap implementations of pmap_ts_referenced() to opportunistically update the page's dirty field. On amd64, use the PDE value already cached in a local variable rather than dereferencing a pointer again and again. Reviewed by: kib, markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D7836
2016-09-10 16:49:25 +00:00
if (PTE_ISMODIFIED(pte))
vm_page_dirty(m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (PTE_ISREFERENCED(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(pv->pv_va);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~PTE_REFERENCED;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Various changes to pmap_ts_referenced() Move PMAP_TS_REFERENCED_MAX out of the various pmap implementations and into vm/pmap.h, and describe what its purpose is. Eliminate the archaic "XXX" comment about its value. I don't believe that its exact value, e.g., 5 versus 6, matters. Update the arm64 and riscv pmap implementations of pmap_ts_referenced() to opportunistically update the page's dirty field. On amd64, use the PDE value already cached in a local variable rather than dereferencing a pointer again and again. Reviewed by: kib, markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D7836
2016-09-10 16:49:25 +00:00
if (++count >= PMAP_TS_REFERENCED_MAX) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pv->pv_pmap);
break;
}
}
}
PMAP_UNLOCK(pv->pv_pmap);
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (count);
}
Implement pmap_unwire(). See r268327 for the motivation behind this change.
2014-07-13 16:27:57 +00:00
/*
* Clear the wired attribute from the mappings for the specified range of
* addresses in the given pmap. Every valid mapping within that range must
* have the wired attribute set. In contrast, invalid mappings cannot have
* the wired attribute set, so they are ignored.
*
* The wired attribute of the page table entry is not a hardware feature, so
* there is no need to invalidate any TLB entries.
*/
static void
mmu_booke_unwire(mmu_t mmu, pmap_t pmap, vm_offset_t sva, vm_offset_t eva)
{
vm_offset_t va;
pte_t *pte;
PMAP_LOCK(pmap);
for (va = sva; va < eva; va += PAGE_SIZE) {
if ((pte = pte_find(mmu, pmap, va)) != NULL &&
PTE_ISVALID(pte)) {
if (!PTE_ISWIRED(pte))
panic("mmu_booke_unwire: pte %p isn't wired",
pte);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~PTE_WIRED;
Implement pmap_unwire(). See r268327 for the motivation behind this change.
2014-07-13 16:27:57 +00:00
pmap->pm_stats.wired_count--;
}
}
PMAP_UNLOCK(pmap);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Return true if the pmap's pv is one of the first 16 pvs linked to from this
* page. This count may be changed upwards or downwards in the future; it is
* only necessary that true be returned for a small subset of pmaps for proper
* page aging.
*/
static boolean_t
mmu_booke_page_exists_quick(mmu_t mmu, pmap_t pmap, vm_page_t m)
{
pv_entry_t pv;
int loops;
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
boolean_t rv;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
("mmu_booke_page_exists_quick: page %p is not managed", m));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
loops = 0;
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
rv = FALSE;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
if (pv->pv_pmap == pmap) {
rv = TRUE;
break;
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (++loops >= 16)
break;
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
return (rv);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Return the number of managed mappings to the given physical page that are
* wired.
*/
static int
mmu_booke_page_wired_mappings(mmu_t mmu, vm_page_t m)
{
pv_entry_t pv;
pte_t *pte;
int count = 0;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
if ((m->oflags & VPO_UNMANAGED) != 0)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (count);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
if ((pte = pte_find(mmu, pv->pv_pmap, pv->pv_va)) != NULL)
if (PTE_ISVALID(pte) && PTE_ISWIRED(pte))
count++;
PMAP_UNLOCK(pv->pv_pmap);
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (count);
}
static int
Fix physical address type to vm_paddr_t.
2012-05-24 21:13:24 +00:00
mmu_booke_dev_direct_mapped(mmu_t mmu, vm_paddr_t pa, vm_size_t size)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
int i;
vm_offset_t va;
/*
* This currently does not work for entries that
* overlap TLB1 entries.
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 0; i < TLB1_ENTRIES; i ++) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (tlb1_iomapped(i, pa, size, &va) == 0)
return (0);
}
return (EFAULT);
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
void
mmu_booke_dumpsys_map(mmu_t mmu, vm_paddr_t pa, size_t sz, void **va)
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
{
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
vm_paddr_t ppa;
vm_offset_t ofs;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
vm_size_t gran;
/* Minidumps are based on virtual memory addresses. */
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
if (do_minidump) {
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
*va = (void *)(vm_offset_t)pa;
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
return;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
/* Raw physical memory dumps don't have a virtual address. */
/* We always map a 256MB page at 256M. */
gran = 256 * 1024 * 1024;
sys: use our roundup2/rounddown2() macros when param.h is available. rounddown2 tends to produce longer lines than the original code and when the code has a high indentation level it was not really advantageous to do the replacement. This tries to strike a balance between readability using the macros and flexibility of having the expressions, so not everything is converted.
2016-04-21 19:57:40 +00:00
ppa = rounddown2(pa, gran);
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
ofs = pa - ppa;
*va = (void *)gran;
tlb1_set_entry((vm_offset_t)va, ppa, gran, _TLB_ENTRY_IO);
if (sz > (gran - ofs))
tlb1_set_entry((vm_offset_t)(va + gran), ppa + gran, gran,
_TLB_ENTRY_IO);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
void
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
mmu_booke_dumpsys_unmap(mmu_t mmu, vm_paddr_t pa, size_t sz, void *va)
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
{
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
vm_paddr_t ppa;
vm_offset_t ofs;
vm_size_t gran;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
int i;
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
/* Minidumps are based on virtual memory addresses. */
/* Nothing to do... */
if (do_minidump)
return;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
break;
}
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
/* Raw physical memory dumps don't have a virtual address. */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
i--;
e.mas1 = 0;
e.mas2 = 0;
e.mas3 = 0;
tlb1_write_entry(&e, i);
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
gran = 256 * 1024 * 1024;
sys: use our roundup2/rounddown2() macros when param.h is available. rounddown2 tends to produce longer lines than the original code and when the code has a high indentation level it was not really advantageous to do the replacement. This tries to strike a balance between readability using the macros and flexibility of having the expressions, so not everything is converted.
2016-04-21 19:57:40 +00:00
ppa = rounddown2(pa, gran);
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
ofs = pa - ppa;
if (sz > (gran - ofs)) {
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
i--;
e.mas1 = 0;
e.mas2 = 0;
e.mas3 = 0;
tlb1_write_entry(&e, i);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
extern struct dump_pa dump_map[PHYS_AVAIL_SZ + 1];
void
mmu_booke_scan_init(mmu_t mmu)
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
{
vm_offset_t va;
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
pte_t *pte;
int i;
if (!do_minidump) {
/* Initialize phys. segments for dumpsys(). */
memset(&dump_map, 0, sizeof(dump_map));
mem_regions(&physmem_regions, &physmem_regions_sz, &availmem_regions,
&availmem_regions_sz);
for (i = 0; i < physmem_regions_sz; i++) {
dump_map[i].pa_start = physmem_regions[i].mr_start;
dump_map[i].pa_size = physmem_regions[i].mr_size;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
return;
}
/* Virtual segments for minidumps: */
memset(&dump_map, 0, sizeof(dump_map));
/* 1st: kernel .data and .bss. */
dump_map[0].pa_start = trunc_page((uintptr_t)_etext);
dump_map[0].pa_size =
round_page((uintptr_t)_end) - dump_map[0].pa_start;
/* 2nd: msgbuf and tables (see pmap_bootstrap()). */
dump_map[1].pa_start = data_start;
dump_map[1].pa_size = data_end - data_start;
/* 3rd: kernel VM. */
va = dump_map[1].pa_start + dump_map[1].pa_size;
/* Find start of next chunk (from va). */
while (va < virtual_end) {
/* Don't dump the buffer cache. */
if (va >= kmi.buffer_sva && va < kmi.buffer_eva) {
va = kmi.buffer_eva;
continue;
}
pte = pte_find(mmu, kernel_pmap, va);
if (pte != NULL && PTE_ISVALID(pte))
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
break;
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
va += PAGE_SIZE;
}
if (va < virtual_end) {
dump_map[2].pa_start = va;
va += PAGE_SIZE;
/* Find last page in chunk. */
while (va < virtual_end) {
/* Don't run into the buffer cache. */
if (va == kmi.buffer_sva)
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
break;
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
pte = pte_find(mmu, kernel_pmap, va);
if (pte == NULL || !PTE_ISVALID(pte))
break;
va += PAGE_SIZE;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
dump_map[2].pa_size = va - dump_map[2].pa_start;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Map a set of physical memory pages into the kernel virtual address space.
* Return a pointer to where it is mapped. This routine is intended to be used
* for mapping device memory, NOT real memory.
*/
static void *
Fix physical address type to vm_paddr_t.
2012-05-24 21:13:24 +00:00
mmu_booke_mapdev(mmu_t mmu, vm_paddr_t pa, vm_size_t size)
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
{
return (mmu_booke_mapdev_attr(mmu, pa, size, VM_MEMATTR_DEFAULT));
}
static void *
mmu_booke_mapdev_attr(mmu_t mmu, vm_paddr_t pa, vm_size_t size, vm_memattr_t ma)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
In mmu_booke_mapdev(), handle mappings that cannot be represented by a single TLB entry. The boot ROM on the MPC85555CDS is 8MB, for example, and in order to map that we need 2 4MB TLB entries.
2008-10-25 03:36:21 +00:00
void *res;
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
uintptr_t va, tmpva;
In mmu_booke_mapdev(), handle mappings that cannot be represented by a single TLB entry. The boot ROM on the MPC85555CDS is 8MB, for example, and in order to map that we need 2 4MB TLB entries.
2008-10-25 03:36:21 +00:00
vm_size_t sz;
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
int i;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
1. Have the APs initialize the TLB1 entries from what has been programmed on the BSP during (early) boot. This makes sure that the APs get configured the same as the BSP, irrspective of how FreeBSD was loaded. 2. Make sure to flush the dcache after writing the TLB1 entries to the boot page. The APs aren't part of the coherency domain just yet. 3. Set pmap_bootstrapped after calling pmap_bootstrap(). The FDT code now maps the devices (like OF), and this resulted in a panic. 4. Since we pre-wire the CCSR, make sure not to map chunks of it in pmap_mapdev().
2012-11-03 22:02:12 +00:00
/*
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
* Check if this is premapped in TLB1. Note: this should probably also
* check whether a sequence of TLB1 entries exist that match the
* requirement, but now only checks the easy case.
1. Have the APs initialize the TLB1 entries from what has been programmed on the BSP during (early) boot. This makes sure that the APs get configured the same as the BSP, irrspective of how FreeBSD was loaded. 2. Make sure to flush the dcache after writing the TLB1 entries to the boot page. The APs aren't part of the coherency domain just yet. 3. Set pmap_bootstrapped after calling pmap_bootstrap(). The FDT code now maps the devices (like OF), and this resulted in a panic. 4. Since we pre-wire the CCSR, make sure not to map chunks of it in pmap_mapdev().
2012-11-03 22:02:12 +00:00
*/
Book-E pmap_mapdev_attr() improvements * Check TLB1 in all mapdev cases, in case the memattr matches an existing mapping (doesn't need to be MAP_DEFAULT). * Fix mapping where the starting address is not a multiple of the widest size base. For instance, it will now properly map 0xffffef000, size 0x11000 using 2 TLB entries, basing it at 0x****f000, instead of 0x***00000. MFC after: 2 weeks
2017-11-10 04:14:48 +00:00
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
continue;
if (pa >= e.phys &&
(pa + size) <= (e.phys + e.size) &&
(ma == VM_MEMATTR_DEFAULT ||
tlb_calc_wimg(pa, ma) ==
(e.mas2 & (MAS2_WIMGE_MASK & ~_TLB_ENTRY_SHARED))))
return (void *)(e.virt +
(vm_offset_t)(pa - e.phys));
1. Have the APs initialize the TLB1 entries from what has been programmed on the BSP during (early) boot. This makes sure that the APs get configured the same as the BSP, irrspective of how FreeBSD was loaded. 2. Make sure to flush the dcache after writing the TLB1 entries to the boot page. The APs aren't part of the coherency domain just yet. 3. Set pmap_bootstrapped after calling pmap_bootstrap(). The FDT code now maps the devices (like OF), and this resulted in a panic. 4. Since we pre-wire the CCSR, make sure not to map chunks of it in pmap_mapdev().
2012-11-03 22:02:12 +00:00
}
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
size = roundup(size, PAGE_SIZE);
In mmu_booke_mapdev(), handle mappings that cannot be represented by a single TLB entry. The boot ROM on the MPC85555CDS is 8MB, for example, and in order to map that we need 2 4MB TLB entries.
2008-10-25 03:36:21 +00:00
Make devices with registers into the KVA region work reliably. Without this, previous KVA allocations (which the PMAP lazily invalidates) in TLB0 could shadow device maps in TLB1. Add a big block comment about some of the caveats with this approach.
2013-10-26 20:57:26 +00:00
/*
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
* The device mapping area is between VM_MAXUSER_ADDRESS and
* VM_MIN_KERNEL_ADDRESS. This gives 1GB of device addressing.
Make devices with registers into the KVA region work reliably. Without this, previous KVA allocations (which the PMAP lazily invalidates) in TLB0 could shadow device maps in TLB1. Add a big block comment about some of the caveats with this approach.
2013-10-26 20:57:26 +00:00
*/
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
#ifdef SPARSE_MAPDEV
/*
* With a sparse mapdev, align to the largest starting region. This
* could feasibly be optimized for a 'best-fit' alignment, but that
* calculation could be very costly.
Book-E pmap_mapdev_attr() improvements * Check TLB1 in all mapdev cases, in case the memattr matches an existing mapping (doesn't need to be MAP_DEFAULT). * Fix mapping where the starting address is not a multiple of the widest size base. For instance, it will now properly map 0xffffef000, size 0x11000 using 2 TLB entries, basing it at 0x****f000, instead of 0x***00000. MFC after: 2 weeks
2017-11-10 04:14:48 +00:00
* Align to the smaller of:
* - first set bit in overlap of (pa & size mask)
* - largest size envelope
*
* It's possible the device mapping may start at a PA that's not larger
* than the size mask, so we need to offset in to maximize the TLB entry
* range and minimize the number of used TLB entries.
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
*/
do {
tmpva = tlb1_map_base;
Book-E pmap_mapdev_attr() improvements * Check TLB1 in all mapdev cases, in case the memattr matches an existing mapping (doesn't need to be MAP_DEFAULT). * Fix mapping where the starting address is not a multiple of the widest size base. For instance, it will now properly map 0xffffef000, size 0x11000 using 2 TLB entries, basing it at 0x****f000, instead of 0x***00000. MFC after: 2 weeks
2017-11-10 04:14:48 +00:00
sz = ffsl(((1 << flsl(size-1)) - 1) & pa);
sz = sz ? min(roundup(sz + 3, 4), flsl(size) - 1) : flsl(size) - 1;
va = roundup(tlb1_map_base, 1 << sz) | (((1 << sz) - 1) & pa);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
} while (!atomic_cmpset_long(&tlb1_map_base, tmpva, va + size));
#else
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
} while (!atomic_cmpset_int(&tlb1_map_base, tmpva, va + size));
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
#else
#ifdef __powerpc64__
va = atomic_fetchadd_long(&tlb1_map_base, size);
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
#else
va = atomic_fetchadd_int(&tlb1_map_base, size);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
#endif
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
res = (void *)va;
In mmu_booke_mapdev(), handle mappings that cannot be represented by a single TLB entry. The boot ROM on the MPC85555CDS is 8MB, for example, and in order to map that we need 2 4MB TLB entries.
2008-10-25 03:36:21 +00:00
do {
sz = 1 << (ilog2(size) & ~1);
Book-E pmap_mapdev_attr() improvements * Check TLB1 in all mapdev cases, in case the memattr matches an existing mapping (doesn't need to be MAP_DEFAULT). * Fix mapping where the starting address is not a multiple of the widest size base. For instance, it will now properly map 0xffffef000, size 0x11000 using 2 TLB entries, basing it at 0x****f000, instead of 0x***00000. MFC after: 2 weeks
2017-11-10 04:14:48 +00:00
/* Align size to PA */
if (pa % sz != 0) {
do {
sz >>= 2;
} while (pa % sz != 0);
}
/* Now align from there to VA */
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
if (va % sz != 0) {
do {
The TLB1 TSIZE is a multiple of 4, not 2, so shift 2 bits, not 1.
2015-08-29 06:52:14 +00:00
sz >>= 2;
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
} while (va % sz != 0);
}
In mmu_booke_mapdev(), handle mappings that cannot be represented by a single TLB entry. The boot ROM on the MPC85555CDS is 8MB, for example, and in order to map that we need 2 4MB TLB entries.
2008-10-25 03:36:21 +00:00
if (bootverbose)
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
printf("Wiring VA=%lx to PA=%jx (size=%lx)\n",
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
va, (uintmax_t)pa, sz);
Book-E pmap_mapdev_attr() improvements * Check TLB1 in all mapdev cases, in case the memattr matches an existing mapping (doesn't need to be MAP_DEFAULT). * Fix mapping where the starting address is not a multiple of the widest size base. For instance, it will now properly map 0xffffef000, size 0x11000 using 2 TLB entries, basing it at 0x****f000, instead of 0x***00000. MFC after: 2 weeks
2017-11-10 04:14:48 +00:00
if (tlb1_set_entry(va, pa, sz,
_TLB_ENTRY_SHARED | tlb_calc_wimg(pa, ma)) < 0)
return (NULL);
In mmu_booke_mapdev(), handle mappings that cannot be represented by a single TLB entry. The boot ROM on the MPC85555CDS is 8MB, for example, and in order to map that we need 2 4MB TLB entries.
2008-10-25 03:36:21 +00:00
size -= sz;
pa += sz;
va += sz;
} while (size > 0);
return (res);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* 'Unmap' a range mapped by mmu_booke_mapdev().
*/
static void
mmu_booke_unmapdev(mmu_t mmu, vm_offset_t va, vm_size_t size)
{
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
#ifdef SUPPORTS_SHRINKING_TLB1
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_offset_t base, offset;
/*
* Unmap only if this is inside kernel virtual space.
*/
if ((va >= VM_MIN_KERNEL_ADDRESS) && (va <= VM_MAX_KERNEL_ADDRESS)) {
base = trunc_page(va);
offset = va & PAGE_MASK;
size = roundup(offset + size, PAGE_SIZE);
Replace kernel virtual address space allocation with vmem. This provides transparent layering and better fragmentation. - Normalize functions that allocate memory to use kmem_* - Those that allocate address space are named kva_* - Those that operate on maps are named kmap_* - Implement recursive allocation handling for kmem_arena in vmem. Reviewed by: alc Tested by: pho Sponsored by: EMC / Isilon Storage Division
2013-08-07 06:21:20 +00:00
kva_free(base, size);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* mmu_booke_object_init_pt preloads the ptes for a given object into the
* specified pmap. This eliminates the blast of soft faults on process startup
* and immediately after an mmap.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
static void
mmu_booke_object_init_pt(mmu_t mmu, pmap_t pmap, vm_offset_t addr,
vm_object_t object, vm_pindex_t pindex, vm_size_t size)
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
Switch the vm_object mutex to be a rwlock. This will enable in the future further optimizations where the vm_object lock will be held in read mode most of the time the page cache resident pool of pages are accessed for reading purposes. The change is mostly mechanical but few notes are reported: * The KPI changes as follow: - VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK() - VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK() - VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK() - VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED() (in order to avoid visibility of implementation details) - The read-mode operations are added: VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(), VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED() * The vm/vm_pager.h namespace pollution avoidance (forcing requiring sys/mutex.h in consumers directly to cater its inlining functions using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h consumers now must include also sys/rwlock.h. * zfs requires a quite convoluted fix to include FreeBSD rwlocks into the compat layer because the name clash between FreeBSD and solaris versions must be avoided. At this purpose zfs redefines the vm_object locking functions directly, isolating the FreeBSD components in specific compat stubs. The KPI results heavilly broken by this commit. Thirdy part ports must be updated accordingly (I can think off-hand of VirtualBox, for example). Sponsored by: EMC / Isilon storage division Reviewed by: jeff Reviewed by: pjd (ZFS specific review) Discussed with: alc Tested by: pho
2013-03-09 02:32:23 +00:00
VM_OBJECT_ASSERT_WLOCKED(object);
Add a new type of VM object: OBJT_SG. An OBJT_SG object is very similar to a device pager (OBJT_DEVICE) object in that it uses fictitious pages to provide aliases to other memory addresses. The primary difference is that it uses an sglist(9) to determine the physical addresses for a given offset into the object instead of invoking the d_mmap() method in a device driver. Reviewed by: alc Approved by: re (kensmith) MFC after: 2 weeks
2009-07-24 13:50:29 +00:00
KASSERT(object->type == OBJT_DEVICE || object->type == OBJT_SG,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
("mmu_booke_object_init_pt: non-device object"));
}
/*
* Perform the pmap work for mincore.
*/
static int
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
mmu_booke_mincore(mmu_t mmu, pmap_t pmap, vm_offset_t addr,
vm_paddr_t *locked_pa)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Do not panic if pmap_mincore() is called. This prevents crashing userland binaries from bringing down the kernel. MFC after: 3 days
2013-11-06 14:36:38 +00:00
/* XXX: this should be implemented at some point */
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (0);
}
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
static int
mmu_booke_change_attr(mmu_t mmu, vm_offset_t addr, vm_size_t sz,
vm_memattr_t mode)
{
vm_offset_t va;
pte_t *pte;
int i, j;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
/* Check TLB1 mappings */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
continue;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
if (addr >= e.virt && addr < e.virt + e.size)
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
break;
}
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
if (i < TLB1_ENTRIES) {
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
/* Only allow full mappings to be modified for now. */
/* Validate the range. */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (j = i, va = addr; va < addr + sz; va += e.size, j++) {
tlb1_read_entry(&e, j);
if (va != e.virt || (sz - (va - addr) < e.size))
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
return (EINVAL);
}
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (va = addr; va < addr + sz; va += e.size, i++) {
tlb1_read_entry(&e, i);
e.mas2 &= ~MAS2_WIMGE_MASK;
e.mas2 |= tlb_calc_wimg(e.phys, mode);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
/*
* Write it out to the TLB. Should really re-sync with other
* cores.
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb1_write_entry(&e, i);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
}
return (0);
}
/* Not in TLB1, try through pmap */
/* First validate the range. */
for (va = addr; va < addr + sz; va += PAGE_SIZE) {
pte = pte_find(mmu, kernel_pmap, va);
if (pte == NULL || !PTE_ISVALID(pte))
return (EINVAL);
}
mtx_lock_spin(&tlbivax_mutex);
tlb_miss_lock();
for (va = addr; va < addr + sz; va += PAGE_SIZE) {
pte = pte_find(mmu, kernel_pmap, va);
*pte &= ~(PTE_MAS2_MASK << PTE_MAS2_SHIFT);
Fix a typo (move parenthesis to correct location in the line). Before this, it would cause the one consumer of this API in powerpc usage (dev/dpaa) to set the PTE WIMG flags to empty instead of --M-, making the cache-enabled buffer portals non-coherent.
2016-12-04 02:15:46 +00:00
*pte |= tlb_calc_wimg(PTE_PA(pte), mode) << PTE_MAS2_SHIFT;
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
tlb0_flush_entry(va);
}
tlb_miss_unlock();
mtx_unlock_spin(&tlbivax_mutex);
Correct the return value for pmap_change_attr() pmap_change_attr() returns an error code, not a paddr. This function is currently unused for powerpc. MFC after: 2 weeks
2017-02-21 05:08:07 +00:00
return (0);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/**************************************************************************/
/* TID handling */
/**************************************************************************/
/*
* Allocate a TID. If necessary, steal one from someone else.
* The new TID is flushed from the TLB before returning.
*/
static tlbtid_t
tid_alloc(pmap_t pmap)
{
tlbtid_t tid;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
int thiscpu;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((pmap != kernel_pmap), ("tid_alloc: kernel pmap"));
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR2(KTR_PMAP, "%s: s (pmap = %p)", __func__, pmap);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
thiscpu = PCPU_GET(cpuid);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tid = PCPU_GET(tid_next);
if (tid > TID_MAX)
tid = TID_MIN;
PCPU_SET(tid_next, tid + 1);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* If we are stealing TID then clear the relevant pmap's field */
if (tidbusy[thiscpu][tid] != NULL) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR2(KTR_PMAP, "%s: warning: stealing tid %d", __func__, tid);
tidbusy[thiscpu][tid]->pm_tid[thiscpu] = TID_NONE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Flush all entries from TLB0 matching this TID. */
Rewrite tid_flush() in C. There's no need for it to be in asm. Also, by writing in C, and marking it static in pmap.c, it saves a branch to the function itself, as it's only used in one location. The generated asm is virtually identical to the handwritten code.
2015-12-30 02:23:14 +00:00
tid_flush(tid);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tidbusy[thiscpu][tid] = pmap;
pmap->pm_tid[thiscpu] = tid;
__asm __volatile("msync; isync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR3(KTR_PMAP, "%s: e (%02d next = %02d)", __func__, tid,
PCPU_GET(tid_next));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
return (tid);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/**************************************************************************/
/* TLB0 handling */
/**************************************************************************/
static void
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
tlb_print_entry(int i, uint32_t mas1, uint64_t mas2, uint32_t mas3,
#else
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb_print_entry(int i, uint32_t mas1, uint32_t mas2, uint32_t mas3,
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
uint32_t mas7)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
int as;
char desc[3];
tlbtid_t tid;
vm_size_t size;
unsigned int tsize;
desc[2] = '\0';
if (mas1 & MAS1_VALID)
desc[0] = 'V';
else
desc[0] = ' ';
if (mas1 & MAS1_IPROT)
desc[1] = 'P';
else
desc[1] = ' ';
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
as = (mas1 & MAS1_TS_MASK) ? 1 : 0;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
tid = MAS1_GETTID(mas1);
tsize = (mas1 & MAS1_TSIZE_MASK) >> MAS1_TSIZE_SHIFT;
size = 0;
if (tsize)
size = tsize2size(tsize);
debugf("%3d: (%s) [AS=%d] "
"sz = 0x%08x tsz = %d tid = %d mas1 = 0x%08x "
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
"mas2(va) = 0x%"PRI0ptrX" mas3(pa) = 0x%08x mas7 = 0x%08x\n",
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
i, desc, as, size, tsize, tid, mas1, mas2, mas3, mas7);
}
/* Convert TLB0 va and way number to tlb0[] table index. */
static inline unsigned int
tlb0_tableidx(vm_offset_t va, unsigned int way)
{
unsigned int idx;
idx = (way * TLB0_ENTRIES_PER_WAY);
idx += (va & MAS2_TLB0_ENTRY_IDX_MASK) >> MAS2_TLB0_ENTRY_IDX_SHIFT;
return (idx);
}
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* Invalidate TLB0 entry.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static inline void
tlb0_flush_entry(vm_offset_t va)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR2(KTR_PMAP, "%s: s va=0x%08x", __func__, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_assert(&tlbivax_mutex, MA_OWNED);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
__asm __volatile("tlbivax 0, %0" :: "r"(va & MAS2_EPN_MASK));
__asm __volatile("isync; msync");
__asm __volatile("tlbsync; msync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR1(KTR_PMAP, "%s: e", __func__);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/* Print out contents of the MAS registers for each TLB0 entry */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
void
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
tlb0_print_tlbentries(void)
{
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
uint32_t mas0, mas1, mas3, mas7;
#ifdef __powerpc64__
uint64_t mas2;
#else
uint32_t mas2;
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
int entryidx, way, idx;
debugf("TLB0 entries:\n");
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
for (way = 0; way < TLB0_WAYS; way ++)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
for (entryidx = 0; entryidx < TLB0_ENTRIES_PER_WAY; entryidx++) {
mas0 = MAS0_TLBSEL(0) | MAS0_ESEL(way);
mtspr(SPR_MAS0, mas0);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
__asm __volatile("isync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mas2 = entryidx << MAS2_TLB0_ENTRY_IDX_SHIFT;
mtspr(SPR_MAS2, mas2);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
__asm __volatile("isync; tlbre");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mas1 = mfspr(SPR_MAS1);
mas2 = mfspr(SPR_MAS2);
mas3 = mfspr(SPR_MAS3);
mas7 = mfspr(SPR_MAS7);
idx = tlb0_tableidx(mas2, way);
tlb_print_entry(idx, mas1, mas2, mas3, mas7);
}
}
/**************************************************************************/
/* TLB1 handling */
/**************************************************************************/
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* TLB1 mapping notes:
*
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
* TLB1[0] Kernel text and data.
* TLB1[1-15] Additional kernel text and data mappings (if required), PCI
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* windows, other devices mappings.
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
/*
* Read an entry from given TLB1 slot.
*/
void
tlb1_read_entry(tlb_entry_t *entry, unsigned int slot)
{
Disable interrupts when updating the TLB Without disabling interrupts it's possible for another thread to preempt and update the registers post-read (tlb1_read_entry) or pre-write (tlb1_write_entry), and confuse the kernel with mixed register states. MFC after: 2 weeks
2017-06-27 01:57:22 +00:00
register_t msr;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
uint32_t mas0;
KASSERT((entry != NULL), ("%s(): Entry is NULL!", __func__));
Disable interrupts when updating the TLB Without disabling interrupts it's possible for another thread to preempt and update the registers post-read (tlb1_read_entry) or pre-write (tlb1_write_entry), and confuse the kernel with mixed register states. MFC after: 2 weeks
2017-06-27 01:57:22 +00:00
msr = mfmsr();
Use the more common Book-E idiom for disabling interrupts. Book-E has the wrteei/wrtee instructions for writing the PSL_EE bit, ignoring all others. Use this instead of the AIM-typical mtmsr. MFC with: r320392
2017-06-30 02:11:32 +00:00
__asm __volatile("wrteei 0");
Disable interrupts when updating the TLB Without disabling interrupts it's possible for another thread to preempt and update the registers post-read (tlb1_read_entry) or pre-write (tlb1_write_entry), and confuse the kernel with mixed register states. MFC after: 2 weeks
2017-06-27 01:57:22 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(slot);
mtspr(SPR_MAS0, mas0);
__asm __volatile("isync; tlbre");
entry->mas1 = mfspr(SPR_MAS1);
entry->mas2 = mfspr(SPR_MAS2);
entry->mas3 = mfspr(SPR_MAS3);
switch ((mfpvr() >> 16) & 0xFFFF) {
case FSL_E500v2:
case FSL_E500mc:
case FSL_E5500:
Take into account mas7/8 when reading/writing TLB entries on e6500 Summary: Current booke/pmap code ignores mas7 and mas8 on e6500 CPU. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7606
2016-08-23 04:26:30 +00:00
case FSL_E6500:
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
entry->mas7 = mfspr(SPR_MAS7);
break;
default:
entry->mas7 = 0;
break;
}
Disable interrupts when updating the TLB Without disabling interrupts it's possible for another thread to preempt and update the registers post-read (tlb1_read_entry) or pre-write (tlb1_write_entry), and confuse the kernel with mixed register states. MFC after: 2 weeks
2017-06-27 01:57:22 +00:00
mtmsr(msr);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
entry->virt = entry->mas2 & MAS2_EPN_MASK;
entry->phys = ((vm_paddr_t)(entry->mas7 & MAS7_RPN) << 32) |
(entry->mas3 & MAS3_RPN);
entry->size =
tsize2size((entry->mas1 & MAS1_TSIZE_MASK) >> MAS1_TSIZE_SHIFT);
}
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
struct tlbwrite_args {
tlb_entry_t *e;
unsigned int idx;
};
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
tlb1_write_entry_int(void *arg)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
struct tlbwrite_args *args = arg;
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
uint32_t mas0;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Select entry */
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(args->idx);
Disable interrupts when updating the TLB Without disabling interrupts it's possible for another thread to preempt and update the registers post-read (tlb1_read_entry) or pre-write (tlb1_write_entry), and confuse the kernel with mixed register states. MFC after: 2 weeks
2017-06-27 01:57:22 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mtspr(SPR_MAS0, mas0);
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
__asm __volatile("isync");
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
mtspr(SPR_MAS1, args->e->mas1);
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
__asm __volatile("isync");
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
mtspr(SPR_MAS2, args->e->mas2);
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
__asm __volatile("isync");
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
mtspr(SPR_MAS3, args->e->mas3);
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
__asm __volatile("isync");
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
switch ((mfpvr() >> 16) & 0xFFFF) {
case FSL_E500mc:
case FSL_E5500:
Take into account mas7/8 when reading/writing TLB entries on e6500 Summary: Current booke/pmap code ignores mas7 and mas8 on e6500 CPU. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7606
2016-08-23 04:26:30 +00:00
case FSL_E6500:
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
mtspr(SPR_MAS8, 0);
__asm __volatile("isync");
/* FALLTHROUGH */
case FSL_E500v2:
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
mtspr(SPR_MAS7, args->e->mas7);
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
__asm __volatile("isync");
break;
default:
break;
}
__asm __volatile("tlbwe; isync; msync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
}
static void
tlb1_write_entry_sync(void *arg)
{
/* Empty synchronization point for smp_rendezvous(). */
}
/*
* Write given entry to TLB1 hardware.
*/
static void
tlb1_write_entry(tlb_entry_t *e, unsigned int idx)
{
struct tlbwrite_args args;
args.e = e;
args.idx = idx;
#ifdef SMP
if ((e->mas2 & _TLB_ENTRY_SHARED) && smp_started) {
mb();
smp_rendezvous(tlb1_write_entry_sync,
tlb1_write_entry_int,
tlb1_write_entry_sync, &args);
} else
#endif
{
register_t msr;
msr = mfmsr();
__asm __volatile("wrteei 0");
tlb1_write_entry_int(&args);
mtmsr(msr);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Return the largest uint value log such that 2^log <= num.
*/
static unsigned int
ilog2(unsigned int num)
{
int lz;
__asm ("cntlzw %0, %1" : "=r" (lz) : "r" (num));
return (31 - lz);
}
/*
* Convert TLB TSIZE value to mapped region size.
*/
static vm_size_t
tsize2size(unsigned int tsize)
{
/*
* size = 4^tsize KB
* size = 4^tsize * 2^10 = 2^(2 * tsize - 10)
*/
return ((1 << (2 * tsize)) * 1024);
}
/*
* Convert region size (must be power of 4) to TLB TSIZE value.
*/
static unsigned int
size2tsize(vm_size_t size)
{
return (ilog2(size) / 2 - 5);
}
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* Register permanent kernel mapping in TLB1.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* Entries are created starting from index 0 (current free entry is
* kept in tlb1_idx) and are not supposed to be invalidated.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
Add support for the Freescale dTSEC DPAA-based ethernet controller. Freescale's QorIQ line includes a new ethernet controller, based on their Datapath Acceleration Architecture (DPAA). This uses a combination of a Frame manager, Buffer manager, and Queue manager to improve performance across all interfaces by being able to pass data directly between hardware acceleration interfaces. As part of this import, Freescale's Netcomm Software (ncsw) driver is imported. This was an attempt by Freescale to create an OS-agnostic sub-driver for managing the hardware, using shims to interface to the OS-specific APIs. This work was abandoned, and Freescale's primary work is in the Linux driver (dual BSD/GPL license). Hence, this was imported directly to sys/contrib, rather than going through the vendor area. Going forward, FreeBSD-specific changes may be made to the ncsw code, diverging from the upstream in potentially incompatible ways. An alternative could be to import the Linux driver itself, using the linuxKPI layer, as that would maintain parity with the vendor-maintained driver. However, the Linux driver has not been evaluated for reliability yet, and may have issues with the import, whereas the ncsw-based driver in this commit was completed by Semihalf 4 years ago, and is very stable. Other SoC modules based on DPAA, which could be added in the future: * Security and Encryption engine (SEC4.x, SEC5.x) * RAID engine Additional work to be done: * Implement polling mode * Test vlan support * Add support for the Pattern Matching Engine, which can do regular expression matching on packets. This driver has been tested on the P5020 QorIQ SoC. Others listed in the dtsec(4) manual page are expected to work as the same DPAA engine is included in all. Obtained from: Semihalf Relnotes: Yes Sponsored by: Alex Perez/Inertial Computing
2016-02-29 03:38:00 +00:00
int
Use the correct type for physical addresses. On Book-E, physical addresses are actually 36-bits, not 32-bits. This is currently worked around by ignoring the top bits. However, in some cases, the boot loader configures CCSR to something above the 32-bit mark. This is stage 1 in updating the pmap to handle 36-bit physaddr.
2015-07-04 19:00:38 +00:00
tlb1_set_entry(vm_offset_t va, vm_paddr_t pa, vm_size_t size,
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
uint32_t flags)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
uint32_t ts, tid;
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
int tsize, index;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (index = 0; index < TLB1_ENTRIES; index++) {
tlb1_read_entry(&e, index);
if ((e.mas1 & MAS1_VALID) == 0)
break;
/* Check if we're just updating the flags, and update them. */
if (e.phys == pa && e.virt == va && e.size == size) {
e.mas2 = (va & MAS2_EPN_MASK) | flags;
tlb1_write_entry(&e, index);
return (0);
}
}
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
if (index >= TLB1_ENTRIES) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
printf("tlb1_set_entry: TLB1 full!\n");
return (-1);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Convert size to TSIZE */
tsize = size2tsize(size);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tid = (TID_KERNEL << MAS1_TID_SHIFT) & MAS1_TID_MASK;
/* XXX TS is hard coded to 0 for now as we only use single address space */
ts = (0 << MAS1_TS_SHIFT) & MAS1_TS_MASK;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
e.phys = pa;
e.virt = va;
e.size = size;
e.mas1 = MAS1_VALID | MAS1_IPROT | ts | tid;
e.mas1 |= ((tsize << MAS1_TSIZE_SHIFT) & MAS1_TSIZE_MASK);
e.mas2 = (va & MAS2_EPN_MASK) | flags;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Set supervisor RWX permission bits */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
e.mas3 = (pa & MAS3_RPN) | MAS3_SR | MAS3_SW | MAS3_SX;
e.mas7 = (pa >> 32) & MAS7_RPN;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb1_write_entry(&e, index);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* XXX in general TLB1 updates should be propagated between CPUs,
* since current design assumes to have the same TLB1 set-up on all
* cores.
*/
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (0);
}
/*
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* Map in contiguous RAM region into the TLB1 using maximum of
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* KERNEL_REGION_MAX_TLB_ENTRIES entries.
*
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* If necessary round up last entry size and return total size
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* used by all allocated entries.
*/
vm_size_t
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
tlb1_mapin_region(vm_offset_t va, vm_paddr_t pa, vm_size_t size)
{
vm_size_t pgs[KERNEL_REGION_MAX_TLB_ENTRIES];
vm_size_t mapped, pgsz, base, mask;
int idx, nents;
/* Round up to the next 1M */
sys: use our roundup2/rounddown2() macros when param.h is available. rounddown2 tends to produce longer lines than the original code and when the code has a high indentation level it was not really advantageous to do the replacement. This tries to strike a balance between readability using the macros and flexibility of having the expressions, so not everything is converted.
2016-04-21 19:57:40 +00:00
size = roundup2(size, 1 << 20);
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
mapped = 0;
idx = 0;
base = va;
pgsz = 64*1024*1024;
while (mapped < size) {
while (mapped < size && idx < KERNEL_REGION_MAX_TLB_ENTRIES) {
while (pgsz > (size - mapped))
pgsz >>= 2;
pgs[idx++] = pgsz;
mapped += pgsz;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
/* We under-map. Correct for this. */
if (mapped < size) {
while (pgs[idx - 1] == pgsz) {
idx--;
mapped -= pgsz;
}
/* XXX We may increase beyond out starting point. */
pgsz <<= 2;
pgs[idx++] = pgsz;
mapped += pgsz;
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
nents = idx;
mask = pgs[0] - 1;
/* Align address to the boundary */
if (va & mask) {
va = (va + mask) & ~mask;
pa = (pa + mask) & ~mask;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
for (idx = 0; idx < nents; idx++) {
pgsz = pgs[idx];
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
debugf("%u: %llx -> %x, size=%x\n", idx, pa, va, pgsz);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb1_set_entry(va, pa, pgsz,
_TLB_ENTRY_SHARED | _TLB_ENTRY_MEM);
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
pa += pgsz;
va += pgsz;
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
mapped = (va - base);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
printf("mapped size 0x%"PRI0ptrX" (wasted space 0x%"PRIxPTR")\n",
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
mapped, mapped - size);
return (mapped);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* TLB1 initialization routine, to be called after the very first
* assembler level setup done in locore.S.
*/
void
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
tlb1_init()
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
uint32_t mas0, mas1, mas2, mas3, mas7;
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
uint32_t tsz;
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
tlb1_get_tlbconf();
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(0);
mtspr(SPR_MAS0, mas0);
__asm __volatile("isync; tlbre");
mas1 = mfspr(SPR_MAS1);
mas2 = mfspr(SPR_MAS2);
mas3 = mfspr(SPR_MAS3);
mas7 = mfspr(SPR_MAS7);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
kernload = ((vm_paddr_t)(mas7 & MAS7_RPN) << 32) |
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
(mas3 & MAS3_RPN);
tsz = (mas1 & MAS1_TSIZE_MASK) >> MAS1_TSIZE_SHIFT;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
kernsize += (tsz > 0) ? tsize2size(tsz) : 0;
Invalidate any TLB1 entries we don't need. The firmware (e.g. U-Boot) may have added entries that conflict with TLB0 entries.
2012-07-02 21:15:56 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Setup TLB miss defaults */
set_mas4_defaults();
}
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
/*
* pmap_early_io_unmap() should be used in short conjunction with
* pmap_early_io_map(), as in the following snippet:
*
* x = pmap_early_io_map(...);
* <do something with x>
* pmap_early_io_unmap(x, size);
*
* And avoiding more allocations between.
*/
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
void
pmap_early_io_unmap(vm_offset_t va, vm_size_t size)
{
int i;
tlb_entry_t e;
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
vm_size_t isize;
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
size = roundup(size, PAGE_SIZE);
isize = size;
for (i = 0; i < TLB1_ENTRIES && size > 0; i++) {
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
continue;
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
if (va <= e.virt && (va + isize) >= (e.virt + e.size)) {
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
size -= e.size;
e.mas1 &= ~MAS1_VALID;
tlb1_write_entry(&e, i);
}
}
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
if (tlb1_map_base == va + isize)
tlb1_map_base -= isize;
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
}
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
vm_offset_t
pmap_early_io_map(vm_paddr_t pa, vm_size_t size)
{
vm_paddr_t pa_base;
vm_offset_t va, sz;
int i;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
KASSERT(!pmap_bootstrapped, ("Do not use after PMAP is up!"));
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
continue;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
if (pa >= e.phys && (pa + size) <=
(e.phys + e.size))
return (e.virt + (pa - e.phys));
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
}
trunc_page() goes through unsigned long, which is too short. sizeof(unsigned long) < sizeof(vm_paddr_t) on Book-E, which uses 36-bit addressing. With this, a CCSR with a physical address above 4GB successfully maps. Sponsored by: Alex Perez/Inertial Computing
2015-11-21 06:03:46 +00:00
pa_base = rounddown(pa, PAGE_SIZE);
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
size = roundup(size + (pa - pa_base), PAGE_SIZE);
Make sure that TLB1 mappings are aligned correctly.
2013-11-17 01:59:42 +00:00
tlb1_map_base = roundup2(tlb1_map_base, 1 << (ilog2(size) & ~1));
Make devices with registers into the KVA region work reliably. Without this, previous KVA allocations (which the PMAP lazily invalidates) in TLB0 could shadow device maps in TLB1. Add a big block comment about some of the caveats with this approach.
2013-10-26 20:57:26 +00:00
va = tlb1_map_base + (pa - pa_base);
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
do {
sz = 1 << (ilog2(size) & ~1);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb1_set_entry(tlb1_map_base, pa_base, sz,
_TLB_ENTRY_SHARED | _TLB_ENTRY_IO);
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
size -= sz;
pa_base += sz;
Make devices with registers into the KVA region work reliably. Without this, previous KVA allocations (which the PMAP lazily invalidates) in TLB0 could shadow device maps in TLB1. Add a big block comment about some of the caveats with this approach.
2013-10-26 20:57:26 +00:00
tlb1_map_base += sz;
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
} while (size > 0);
return (va);
}
Simplify the page tracking for VA<->PA translations. Drop the tracking down to the pmap layer, with optimizations to only track necessary pages. This should give a (slight) performance improvement, as well as a stability improvement, as the tracking is already mostly handled by the pmap layer.
2016-11-16 05:24:42 +00:00
void
pmap_track_page(pmap_t pmap, vm_offset_t va)
{
vm_paddr_t pa;
vm_page_t page;
struct pv_entry *pve;
Use trunc_page() instead of rolling my own in pmap_track_page()
2016-12-05 02:27:50 +00:00
va = trunc_page(va);
Simplify the page tracking for VA<->PA translations. Drop the tracking down to the pmap layer, with optimizations to only track necessary pages. This should give a (slight) performance improvement, as well as a stability improvement, as the tracking is already mostly handled by the pmap layer.
2016-11-16 05:24:42 +00:00
pa = pmap_kextract(va);
rw_wlock(&pvh_global_lock);
PMAP_LOCK(pmap);
page = PHYS_TO_VM_PAGE(pa);
TAILQ_FOREACH(pve, &page->md.pv_list, pv_link) {
if ((pmap == pve->pv_pmap) && (va == pve->pv_va)) {
goto out;
}
}
page->md.pv_tracked = true;
pv_insert(pmap, va, page);
out:
PMAP_UNLOCK(pmap);
rw_wunlock(&pvh_global_lock);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Setup MAS4 defaults.
* These values are loaded to MAS0-2 on a TLB miss.
*/
static void
set_mas4_defaults(void)
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint32_t mas4;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Defaults: TLB0, PID0, TSIZED=4K */
mas4 = MAS4_TLBSELD0;
mas4 |= (TLB_SIZE_4K << MAS4_TSIZED_SHIFT) & MAS4_TSIZED_MASK;
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
#ifdef SMP
mas4 |= MAS4_MD;
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mtspr(SPR_MAS4, mas4);
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
__asm __volatile("isync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Print out contents of the MAS registers for each TLB1 entry
*/
void
tlb1_print_tlbentries(void)
{
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
uint32_t mas0, mas1, mas3, mas7;
#ifdef __powerpc64__
uint64_t mas2;
#else
uint32_t mas2;
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
int i;
debugf("TLB1 entries:\n");
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
for (i = 0; i < TLB1_ENTRIES; i++) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(i);
mtspr(SPR_MAS0, mas0);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
__asm __volatile("isync; tlbre");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mas1 = mfspr(SPR_MAS1);
mas2 = mfspr(SPR_MAS2);
mas3 = mfspr(SPR_MAS3);
mas7 = mfspr(SPR_MAS7);
tlb_print_entry(i, mas1, mas2, mas3, mas7);
}
}
/*
* Return 0 if the physical IO range is encompassed by one of the
* the TLB1 entries, otherwise return related error code.
*/
static int
tlb1_iomapped(int i, vm_paddr_t pa, vm_size_t size, vm_offset_t *va)
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint32_t prot;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_paddr_t pa_start;
vm_paddr_t pa_end;
unsigned int entry_tsize;
vm_size_t entry_size;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*va = (vm_offset_t)NULL;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb1_read_entry(&e, i);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Skip invalid entries */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
if (!(e.mas1 & MAS1_VALID))
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (EINVAL);
/*
* The entry must be cache-inhibited, guarded, and r/w
* so it can function as an i/o page
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
prot = e.mas2 & (MAS2_I | MAS2_G);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (prot != (MAS2_I | MAS2_G))
return (EPERM);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
prot = e.mas3 & (MAS3_SR | MAS3_SW);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (prot != (MAS3_SR | MAS3_SW))
return (EPERM);
/* The address should be within the entry range. */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
entry_tsize = (e.mas1 & MAS1_TSIZE_MASK) >> MAS1_TSIZE_SHIFT;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((entry_tsize), ("tlb1_iomapped: invalid entry tsize"));
entry_size = tsize2size(entry_tsize);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
pa_start = (((vm_paddr_t)e.mas7 & MAS7_RPN) << 32) |
(e.mas3 & MAS3_RPN);
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
pa_end = pa_start + entry_size;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if ((pa < pa_start) || ((pa + size) > pa_end))
return (ERANGE);
/* Return virtual address of this mapping. */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
*va = (e.mas2 & MAS2_EPN_MASK) + (pa - pa_start);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (0);
}
Rewrite tid_flush() in C. There's no need for it to be in asm. Also, by writing in C, and marking it static in pmap.c, it saves a branch to the function itself, as it's only used in one location. The generated asm is virtually identical to the handwritten code.
2015-12-30 02:23:14 +00:00
/*
* Invalidate all TLB0 entries which match the given TID. Note this is
* dedicated for cases when invalidations should NOT be propagated to other
* CPUs.
*/
static void
tid_flush(tlbtid_t tid)
{
register_t msr;
uint32_t mas0, mas1, mas2;
int entry, way;
/* Don't evict kernel translations */
if (tid == TID_KERNEL)
return;
msr = mfmsr();
__asm __volatile("wrteei 0");
for (way = 0; way < TLB0_WAYS; way++)
for (entry = 0; entry < TLB0_ENTRIES_PER_WAY; entry++) {
mas0 = MAS0_TLBSEL(0) | MAS0_ESEL(way);
mtspr(SPR_MAS0, mas0);
__asm __volatile("isync");
mas2 = entry << MAS2_TLB0_ENTRY_IDX_SHIFT;
mtspr(SPR_MAS2, mas2);
__asm __volatile("isync; tlbre");
mas1 = mfspr(SPR_MAS1);
if (!(mas1 & MAS1_VALID))
continue;
if (((mas1 & MAS1_TID_MASK) >> MAS1_TID_SHIFT) != tid)
continue;
mas1 &= ~MAS1_VALID;
mtspr(SPR_MAS1, mas1);
__asm __volatile("isync; tlbwe; isync; msync");
}
mtmsr(msr);
}
Reference in New Issue Copy Permalink