d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
dc42509049
freebsd-dev/sys/riscv/include/cpufunc.h

136 lines
3.4 KiB
C
Raw Normal View History

Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
/*-
Update RISC-V port to Privileged Architecture Version 1.9. Sponsored by: DARPA, AFRL Sponsored by: HEIF5
2016-08-02 14:50:14 +00:00
* Copyright (c) 2015-2016 Ruslan Bukin <br@bsdpad.com>
Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
* All rights reserved.
*
* Portions of this software were developed by SRI International and the
* University of Cambridge Computer Laboratory under DARPA/AFRL contract
* FA8750-10-C-0237 ("CTSRD"), as part of the DARPA CRASH research programme.
*
* Portions of this software were developed by the University of Cambridge
* Computer Laboratory as part of the CTSRD Project, with support from the
* UK Higher Education Innovation Fund (HEIF).
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $FreeBSD$
*/
#ifndef _MACHINE_CPUFUNC_H_
#define _MACHINE_CPUFUNC_H_
static __inline void
breakpoint(void)
{
__asm("ebreak");
}
Export a breakpoint() function to userland for riscv. As a result, enable tests using breakpoint() on riscv. Reviewed by: br Differential Revision: https://reviews.freebsd.org/D15191
2018-05-16 16:56:35 +00:00
#ifdef _KERNEL
#include <machine/riscvreg.h>
Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
static __inline register_t
intr_disable(void)
{
uint64_t ret;
__asm __volatile(
Update RISC-V port to Privileged Architecture Version 1.9. Sponsored by: DARPA, AFRL Sponsored by: HEIF5
2016-08-02 14:50:14 +00:00
"csrrci %0, sstatus, %1"
: "=&r" (ret) : "i" (SSTATUS_SIE)
Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
);
Update RISC-V port to Privileged Architecture Version 1.9. Sponsored by: DARPA, AFRL Sponsored by: HEIF5
2016-08-02 14:50:14 +00:00
return (ret & (SSTATUS_SIE));
Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
}
static __inline void
intr_restore(register_t s)
{
__asm __volatile(
"csrs sstatus, %0"
:: "r" (s)
);
}
static __inline void
intr_enable(void)
{
__asm __volatile(
Update RISC-V port to Privileged Architecture Version 1.9. Sponsored by: DARPA, AFRL Sponsored by: HEIF5
2016-08-02 14:50:14 +00:00
"csrsi sstatus, %0"
:: "i" (SSTATUS_SIE)
Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
);
}
Various fixes for TLB management on RISC-V. - Remove the arm64-specific cpu_*cache* and cpu_tlb_flush* functions. Instead, add RISC-V specific inline functions in cpufunc.h for the fence.i and sfence.vma instructions. - Catch up to changes in the arm64 pmap and remove all the cpu_dcache_* calls, pmap_is_current, pmap_l3_valid_cacheable, and PTE_NEXT bits from pmap. - Remove references to the unimplemented riscv_setttb(). - Remove unused cpu_nullop. - Add a link to the SBI doc to sbi.h. - Add support for a 4th argument in SBI calls. It's not documented but it seems implied for the asid argument to SBI_REMOVE_SFENCE_VMA_ASID. - Pass the arguments from sbi_remote_sfence*() to the SEE. BBL ignores them so this is just cosmetic. - Flush icaches on other CPUs when they resume from kdb in case the debugger wrote any breakpoints while the CPUs were paused in the IPI_STOP handler. - Add SMP vs UP versions of pmap_invalidate_* similar to amd64. The UP versions just use simple fences. The SMP versions use the sbi_remove_sfence*() functions to perform TLB shootdowns. Since we don't have a valid pm_active field in the riscv pmap, just IPI all CPUs for all invalidations for now. - Remove an extraneous TLB flush from the end of pmap_bootstrap(). - Don't do a TLB flush when writing new mappings in pmap_enter(), only if modifying an existing mapping. Note that for COW faults a TLB flush is only performed after explicitly clearing the old mapping as is done in other pmaps. - Sync the i-cache on all harts before updating the PTE for executable mappings in pmap_enter and pmap_enter_quick. Previously the i-cache was only sync'd after updating the PTE in pmap_enter. - Use sbi_remote_fence() instead of smp_rendezvous in pmap_sync_icache(). Reviewed by: markj Approved by: re (gjb, kib) Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D17414
2018-10-15 18:56:54 +00:00
/* NB: fence() is defined as a macro in <machine/atomic.h>. */
static __inline void
fence_i(void)
{
Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
Various fixes for TLB management on RISC-V. - Remove the arm64-specific cpu_*cache* and cpu_tlb_flush* functions. Instead, add RISC-V specific inline functions in cpufunc.h for the fence.i and sfence.vma instructions. - Catch up to changes in the arm64 pmap and remove all the cpu_dcache_* calls, pmap_is_current, pmap_l3_valid_cacheable, and PTE_NEXT bits from pmap. - Remove references to the unimplemented riscv_setttb(). - Remove unused cpu_nullop. - Add a link to the SBI doc to sbi.h. - Add support for a 4th argument in SBI calls. It's not documented but it seems implied for the asid argument to SBI_REMOVE_SFENCE_VMA_ASID. - Pass the arguments from sbi_remote_sfence*() to the SEE. BBL ignores them so this is just cosmetic. - Flush icaches on other CPUs when they resume from kdb in case the debugger wrote any breakpoints while the CPUs were paused in the IPI_STOP handler. - Add SMP vs UP versions of pmap_invalidate_* similar to amd64. The UP versions just use simple fences. The SMP versions use the sbi_remove_sfence*() functions to perform TLB shootdowns. Since we don't have a valid pm_active field in the riscv pmap, just IPI all CPUs for all invalidations for now. - Remove an extraneous TLB flush from the end of pmap_bootstrap(). - Don't do a TLB flush when writing new mappings in pmap_enter(), only if modifying an existing mapping. Note that for COW faults a TLB flush is only performed after explicitly clearing the old mapping as is done in other pmaps. - Sync the i-cache on all harts before updating the PTE for executable mappings in pmap_enter and pmap_enter_quick. Previously the i-cache was only sync'd after updating the PTE in pmap_enter. - Use sbi_remote_fence() instead of smp_rendezvous in pmap_sync_icache(). Reviewed by: markj Approved by: re (gjb, kib) Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D17414
2018-10-15 18:56:54 +00:00
__asm __volatile("fence.i" ::: "memory");
}
Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
Various fixes for TLB management on RISC-V. - Remove the arm64-specific cpu_*cache* and cpu_tlb_flush* functions. Instead, add RISC-V specific inline functions in cpufunc.h for the fence.i and sfence.vma instructions. - Catch up to changes in the arm64 pmap and remove all the cpu_dcache_* calls, pmap_is_current, pmap_l3_valid_cacheable, and PTE_NEXT bits from pmap. - Remove references to the unimplemented riscv_setttb(). - Remove unused cpu_nullop. - Add a link to the SBI doc to sbi.h. - Add support for a 4th argument in SBI calls. It's not documented but it seems implied for the asid argument to SBI_REMOVE_SFENCE_VMA_ASID. - Pass the arguments from sbi_remote_sfence*() to the SEE. BBL ignores them so this is just cosmetic. - Flush icaches on other CPUs when they resume from kdb in case the debugger wrote any breakpoints while the CPUs were paused in the IPI_STOP handler. - Add SMP vs UP versions of pmap_invalidate_* similar to amd64. The UP versions just use simple fences. The SMP versions use the sbi_remove_sfence*() functions to perform TLB shootdowns. Since we don't have a valid pm_active field in the riscv pmap, just IPI all CPUs for all invalidations for now. - Remove an extraneous TLB flush from the end of pmap_bootstrap(). - Don't do a TLB flush when writing new mappings in pmap_enter(), only if modifying an existing mapping. Note that for COW faults a TLB flush is only performed after explicitly clearing the old mapping as is done in other pmaps. - Sync the i-cache on all harts before updating the PTE for executable mappings in pmap_enter and pmap_enter_quick. Previously the i-cache was only sync'd after updating the PTE in pmap_enter. - Use sbi_remote_fence() instead of smp_rendezvous in pmap_sync_icache(). Reviewed by: markj Approved by: re (gjb, kib) Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D17414
2018-10-15 18:56:54 +00:00
static __inline void
sfence_vma(void)
{
Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
Various fixes for TLB management on RISC-V. - Remove the arm64-specific cpu_*cache* and cpu_tlb_flush* functions. Instead, add RISC-V specific inline functions in cpufunc.h for the fence.i and sfence.vma instructions. - Catch up to changes in the arm64 pmap and remove all the cpu_dcache_* calls, pmap_is_current, pmap_l3_valid_cacheable, and PTE_NEXT bits from pmap. - Remove references to the unimplemented riscv_setttb(). - Remove unused cpu_nullop. - Add a link to the SBI doc to sbi.h. - Add support for a 4th argument in SBI calls. It's not documented but it seems implied for the asid argument to SBI_REMOVE_SFENCE_VMA_ASID. - Pass the arguments from sbi_remote_sfence*() to the SEE. BBL ignores them so this is just cosmetic. - Flush icaches on other CPUs when they resume from kdb in case the debugger wrote any breakpoints while the CPUs were paused in the IPI_STOP handler. - Add SMP vs UP versions of pmap_invalidate_* similar to amd64. The UP versions just use simple fences. The SMP versions use the sbi_remove_sfence*() functions to perform TLB shootdowns. Since we don't have a valid pm_active field in the riscv pmap, just IPI all CPUs for all invalidations for now. - Remove an extraneous TLB flush from the end of pmap_bootstrap(). - Don't do a TLB flush when writing new mappings in pmap_enter(), only if modifying an existing mapping. Note that for COW faults a TLB flush is only performed after explicitly clearing the old mapping as is done in other pmaps. - Sync the i-cache on all harts before updating the PTE for executable mappings in pmap_enter and pmap_enter_quick. Previously the i-cache was only sync'd after updating the PTE in pmap_enter. - Use sbi_remote_fence() instead of smp_rendezvous in pmap_sync_icache(). Reviewed by: markj Approved by: re (gjb, kib) Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D17414
2018-10-15 18:56:54 +00:00
__asm __volatile("sfence.vma" ::: "memory");
}
static __inline void
sfence_vma_page(uintptr_t addr)
{
__asm __volatile("sfence.vma %0" :: "r" (addr) : "memory");
}
RISC-V: Add macros for reading performance counter CSRs. The RISC-V spec defines several performance counter CSRs such as: cycle, time, instret, hpmcounter(3...31). They are defined to be 64-bits wide on all RISC-V architectures. On RV64 and RV128 they can be read from a single CSR. On RV32, additional CSRs (given the suffix "h") are present which contain the upper 32 bits of these counters, and must be read as well. (See section 2.8 in the User ISA Spec for full details.) This change adds macros for reading these values safely on any RISC-V ISA length. Obviously we aren't supporting anything other than RV64 at the moment, but this ensures we won't need to change how we read these values if we ever do. Submitted by: Mitchell Horne <mhorne063@gmail.com> Reviewed by: jhb MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D17952
2018-11-13 18:12:06 +00:00
#define rdcycle() csr_read64(cycle)
#define rdtime() csr_read64(time)
#define rdinstret() csr_read64(instret)
#define rdhpmcounter(n) csr_read64(hpmcounter##n)
Provide a template for busdma code for RISC-V. RISC-V ISA specifies no cache management instructions so leave cache operations in cpufunc.h as no-op for now. Note some new hardware comes with their own memory-mapped cache management controller. Tested on HiFive Unleashed board with cgem(4). Reviewed by: markj Obtained from: arm64 Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D20126
2019-05-07 13:41:43 +00:00
extern int64_t dcache_line_size;
extern int64_t icache_line_size;
#define cpu_dcache_wbinv_range(a, s)
#define cpu_dcache_inv_range(a, s)
#define cpu_dcache_wb_range(a, s)
#define cpu_idcache_wbinv_range(a, s)
#define cpu_icache_sync_range(a, s)
#define cpu_icache_sync_range_checked(a, s)
Rename sptbr to satp per v1.10 of the privileged architecture spec. Add a subroutine for updating satp, for use when updating the active pmap. No functional change intended. Reviewed by: jhb MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D18462
2018-12-07 15:55:23 +00:00
static __inline void
load_satp(uint64_t val)
{
__asm __volatile("csrw satp, %0" :: "r"(val));
}
Various fixes for TLB management on RISC-V. - Remove the arm64-specific cpu_*cache* and cpu_tlb_flush* functions. Instead, add RISC-V specific inline functions in cpufunc.h for the fence.i and sfence.vma instructions. - Catch up to changes in the arm64 pmap and remove all the cpu_dcache_* calls, pmap_is_current, pmap_l3_valid_cacheable, and PTE_NEXT bits from pmap. - Remove references to the unimplemented riscv_setttb(). - Remove unused cpu_nullop. - Add a link to the SBI doc to sbi.h. - Add support for a 4th argument in SBI calls. It's not documented but it seems implied for the asid argument to SBI_REMOVE_SFENCE_VMA_ASID. - Pass the arguments from sbi_remote_sfence*() to the SEE. BBL ignores them so this is just cosmetic. - Flush icaches on other CPUs when they resume from kdb in case the debugger wrote any breakpoints while the CPUs were paused in the IPI_STOP handler. - Add SMP vs UP versions of pmap_invalidate_* similar to amd64. The UP versions just use simple fences. The SMP versions use the sbi_remove_sfence*() functions to perform TLB shootdowns. Since we don't have a valid pm_active field in the riscv pmap, just IPI all CPUs for all invalidations for now. - Remove an extraneous TLB flush from the end of pmap_bootstrap(). - Don't do a TLB flush when writing new mappings in pmap_enter(), only if modifying an existing mapping. Note that for COW faults a TLB flush is only performed after explicitly clearing the old mapping as is done in other pmaps. - Sync the i-cache on all harts before updating the PTE for executable mappings in pmap_enter and pmap_enter_quick. Previously the i-cache was only sync'd after updating the PTE in pmap_enter. - Use sbi_remote_fence() instead of smp_rendezvous in pmap_sync_icache(). Reviewed by: markj Approved by: re (gjb, kib) Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D17414
2018-10-15 18:56:54 +00:00
#define cpufunc_nullop() riscv_nullop()
Import RISC-V machine headers. This is a minimal set required to compile kernel and userland. Reviewed by: andrew, imp, kib Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4554
2015-12-17 18:44:30 +00:00
void riscv_nullop(void);
#endif /* _KERNEL */
#endif /* _MACHINE_CPUFUNC_H_ */
Reference in New Issue Copy Permalink