From 48d2c7cc305cc94542f95d3c2f378f63b92c8562 Mon Sep 17 00:00:00 2001 From: Mark Johnston Date: Tue, 13 Apr 2021 17:39:19 -0400 Subject: [PATCH] Add the KASAN runtime KASAN enables the use of LLVM's AddressSanitizer in the kernel. This feature makes use of compiler instrumentation to validate memory accesses in the kernel and detect several types of bugs, including use-after-frees and out-of-bounds accesses. It is particularly effective when combined with test suites or syzkaller. KASAN has high CPU and memory usage overhead and so is not suited for production environments. The runtime and pmap maintain a shadow of the kernel map to store information about the validity of memory mapped at a given kernel address. The runtime implements a number of functions defined by the compiler ABI. These are prefixed by __asan. The compiler emits calls to __asan_load*() and __asan_store*() around memory accesses, and the runtime consults the shadow map to determine whether a given access is valid. kasan_mark() is called by various kernel allocators to update state in the shadow map. Updates to those allocators will come in subsequent commits. The runtime also defines various interceptors. Some low-level routines are implemented in assembly and are thus not amenable to compiler instrumentation. To handle this, the runtime implements these routines on behalf of the rest of the kernel. The sanitizer implementation validates memory accesses manually before handing off to the real implementation. The sanitizer in a KASAN-configured kernel can be disabled by setting the loader tunable debug.kasan.disable=1. Obtained from: NetBSD Sponsored by: The FreeBSD Foundation (cherry picked from commit 38da497a4dfcf1979c8c2b0e9f3fa0564035c147) --- share/man/man9/Makefile | 3 + share/man/man9/kasan.9 | 171 ++++++ sys/conf/files | 2 + sys/kern/subr_asan.c | 1091 +++++++++++++++++++++++++++++++++++++++ sys/sys/asan.h | 68 +++ 5 files changed, 1335 insertions(+) create mode 100644 share/man/man9/kasan.9 create mode 100644 sys/kern/subr_asan.c create mode 100644 sys/sys/asan.h diff --git a/share/man/man9/Makefile b/share/man/man9/Makefile index 8793eb6b7278..500cd252a3df 100644 --- a/share/man/man9/Makefile +++ b/share/man/man9/Makefile @@ -186,6 +186,7 @@ MAN= accept_filter.9 \ insmntque.9 \ intro.9 \ ithread.9 \ + kasan.9 \ KASSERT.9 \ kern_reboot.9 \ kern_testfrwk.9 \ @@ -1314,6 +1315,8 @@ MLINKS+=kernel_mount.9 free_mntarg.9 \ kernel_mount.9 mount_argb.9 \ kernel_mount.9 mount_argf.9 \ kernel_mount.9 mount_argsu.9 +MLINKS+=kasan.9 KASAN.9 \ + kasan.9 kasan_mark.9 MLINKS+=khelp.9 khelp_add_hhook.9 \ khelp.9 KHELP_DECLARE_MOD.9 \ khelp.9 KHELP_DECLARE_MOD_UMA.9 \ diff --git a/share/man/man9/kasan.9 b/share/man/man9/kasan.9 new file mode 100644 index 000000000000..ecc068209913 --- /dev/null +++ b/share/man/man9/kasan.9 @@ -0,0 +1,171 @@ +.\"- +.\" Copyright (c) 2021 The FreeBSD Foundation +.\" +.\" This documentation was written by Mark Johnston under sponsorship from +.\" the FreeBSD Foundation. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd April 13, 2021 +.Dt KASAN 9 +.Os +.Sh NAME +.Nm kasan +.Nd kernel address sanitizer +.Sh SYNOPSIS +To compile KASAN into the kernel, place the following line in your kernel +configuration file: +.Bd -ragged -offset indent +.Cd "options KASAN" +.Ed +.Pp +.Ft void +.Fn kasan_mark "const void *addr" "size_t size" "size_t redzsize" "uint8_t code" +.Sh DESCRIPTION +.Nm +is a subsystem which leverages compiler instrumentation to detect invalid +memory accesses in the kernel. +Currently it is implemented only on the amd64 platform. +.Pp +When +.Nm +is compiled into the kernel, the compiler is configured to emit function +calls upon every memory access. +The functions are implemented by +.Nm +and permit run-time detection of several types of bugs including +use-after-frees, double frees and frees of invalid pointers, and out-of-bounds +accesses. +These protections apply to memory allocated by +.Xr uma 9 , +.Xr malloc 9 +and related functions, and +.Fn kmem_malloc +and related functions, +as well as global variables and kernel stacks. +.Nm +is conservative and will not detect all instances of these types of bugs. +Memory accesses through the kernel map are sanitized, but accesses via the +direct map are not. +When +.Nm +is configured, the kernel aims to minimize its use of the direct map. +.Sh IMPLEMENTATION NOTES +.Nm +is implemented using compiler instrumentation and a kernel runtime. +When a +kernel is built with the KASAN option enabled, the compiler inserts function calls +before most memory accesses in the generated code. +The runtime implements the corresponding functions, which decide whether a +given access is valid. +If not, the runtime prints a warning or panics the kernel, depending on the +value of the +.Sy debug.kasan.panic_on_violation +sysctl/tunable. +.Pp +The +.Nm +runtime works by maintaining a shadow map for the kernel map. +There exists a linear mapping between addresses in the kernel map and addresses +in the shadow map. +The shadow map is used to store information about the current state of +allocations from the kernel map. +For example, when a buffer is returned by +.Xr malloc 9 , +the corresponding region of the shadow map is marked to indicate that the +buffer is valid. +When it is freed, the shadow map is updated to mark the buffer as invalid. +Accesses to the buffer are intercepted by the +.Nm +runtime and validated using the contents of the shadow map. +.Pp +Upon booting, all kernel memory is marked as valid. +Kernel allocators must mark cached but free buffers as invalid, and must mark +them valid before freeing the kernel virtual address range. +This slightly reduces the effectiveness of +.Nm +but simplifies its maintenance and integration into the kernel. +.Pp +Updates to the shadow map are performed by calling +.Fn kasan_mark . +Parameter +.Fa addr +is the address of the buffer whose shadow is to be updated, +.Fa size +is the usable size of the buffer, and +.Fa redzsize +is the full size of the buffer allocated from lower layers of the system. +.Fa redzsize +must be greater than or equal to +.Fa size . +In some cases kernel allocators will return a buffer larger than that requested +by the consumer; the unused space at the end is referred to as a red zone and is +always marked as invalid. +.Fa code +allows the caller to specify an identifier used when marking a buffer as invalid. +The identifier is included in any reports generated by +.Nm +and helps identify the source of the invalid access. +For instance, when an item is freed to a +.Xr uma 9 +zone, the item is marked with +.Dv KASAN_UMA_FREED . +See +.In sys/asan.h +for the available identifiers. +If the entire buffer is to be marked valid, i.e., +.Fa size +and +.Fa redzsize +are equal, +.Fa code +should be 0. +.Sh SEE ALSO +.Xr malloc 9 , +.Xr memguard 9 , +.Xr redzone 9 , +.Xr uma 9 +.Sh HISTORY +.Nm +first appeared in +.Fx 14.0 . +.Sh BUGS +Accesses to kernel memory outside of the kernel map are ignored by the +.Nm +runtime. +When +.Nm +is configured, the kernel memory allocators are configured to use the kernel +map, but some uses of the direct map remain. +For example, on amd64, accesses to page table pages are not tracked. +.Pp +Some kernel memory allocators explicitly permit accesses after an object has +been freed. +These cannot be sanitized by +.Nm . +For example, memory from all +.Xr uma 9 +zones initialized with the +.Dv UMA_ZONE_NOFREE +flag are not sanitized. diff --git a/sys/conf/files b/sys/conf/files index a5f23ca09774..282a86d74347 100644 --- a/sys/conf/files +++ b/sys/conf/files @@ -3900,6 +3900,8 @@ kern/stack_protector.c standard \ compile-with "${NORMAL_C:N-fstack-protector*}" kern/subr_acl_nfs4.c optional ufs_acl | zfs kern/subr_acl_posix1e.c optional ufs_acl +kern/subr_asan.c optional kasan \ + compile-with "${NORMAL_C:N-fsanitize*}" kern/subr_autoconf.c standard kern/subr_blist.c standard kern/subr_boot.c standard diff --git a/sys/kern/subr_asan.c b/sys/kern/subr_asan.c new file mode 100644 index 000000000000..842370ad1e63 --- /dev/null +++ b/sys/kern/subr_asan.c @@ -0,0 +1,1091 @@ +/* $NetBSD: subr_asan.c,v 1.26 2020/09/10 14:10:46 maxv Exp $ */ + +/* + * Copyright (c) 2018-2020 Maxime Villard, m00nbsd.net + * All rights reserved. + * + * This code is part of the KASAN subsystem of the NetBSD kernel. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#define SAN_RUNTIME + +#include +__FBSDID("$FreeBSD$"); +#if 0 +__KERNEL_RCSID(0, "$NetBSD: subr_asan.c,v 1.26 2020/09/10 14:10:46 maxv Exp $"); +#endif + +#include +#include +#include +#include +#include +#include + +#include + +/* ASAN constants. Part of the compiler ABI. */ +#define KASAN_SHADOW_MASK (KASAN_SHADOW_SCALE - 1) +#define KASAN_ALLOCA_SCALE_SIZE 32 + +/* ASAN ABI version. */ +#if defined(__clang__) && (__clang_major__ - 0 >= 6) +#define ASAN_ABI_VERSION 8 +#elif __GNUC_PREREQ__(7, 1) && !defined(__clang__) +#define ASAN_ABI_VERSION 8 +#elif __GNUC_PREREQ__(6, 1) && !defined(__clang__) +#define ASAN_ABI_VERSION 6 +#else +#error "Unsupported compiler version" +#endif + +#define __RET_ADDR (unsigned long)__builtin_return_address(0) + +/* Global variable descriptor. Part of the compiler ABI. */ +struct __asan_global_source_location { + const char *filename; + int line_no; + int column_no; +}; + +struct __asan_global { + const void *beg; /* address of the global variable */ + size_t size; /* size of the global variable */ + size_t size_with_redzone; /* size with the redzone */ + const void *name; /* name of the variable */ + const void *module_name; /* name of the module where the var is declared */ + unsigned long has_dynamic_init; /* the var has dyn initializer (c++) */ + struct __asan_global_source_location *location; +#if ASAN_ABI_VERSION >= 7 + uintptr_t odr_indicator; /* the address of the ODR indicator symbol */ +#endif +}; + +FEATURE(kasan, "Kernel address sanitizer"); + +static SYSCTL_NODE(_debug, OID_AUTO, kasan, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, + "KASAN options"); + +static int panic_on_violation = 1; +SYSCTL_INT(_debug_kasan, OID_AUTO, panic_on_violation, CTLFLAG_RDTUN, + &panic_on_violation, 0, + "Panic if an invalid access is detected"); + +static bool kasan_enabled __read_mostly = false; + +/* -------------------------------------------------------------------------- */ + +void +kasan_shadow_map(void *addr, size_t size) +{ + size_t sz, npages, i; + vm_offset_t sva, eva; + + KASSERT((vm_offset_t)addr % KASAN_SHADOW_SCALE == 0, + ("%s: invalid address %p", __func__, addr)); + + sz = roundup(size, KASAN_SHADOW_SCALE) / KASAN_SHADOW_SCALE; + + sva = kasan_md_addr_to_shad((vm_offset_t)addr); + eva = kasan_md_addr_to_shad((vm_offset_t)addr) + sz; + + sva = rounddown(sva, PAGE_SIZE); + eva = roundup(eva, PAGE_SIZE); + + npages = (eva - sva) / PAGE_SIZE; + + KASSERT(sva >= KASAN_MIN_ADDRESS && eva < KASAN_MAX_ADDRESS, + ("%s: invalid address range %#lx-%#lx", __func__, sva, eva)); + + for (i = 0; i < npages; i++) + pmap_kasan_enter(sva + ptoa(i)); +} + +void +kasan_init(void) +{ + int disabled; + + disabled = 0; + TUNABLE_INT_FETCH("debug.kasan.disabled", &disabled); + if (disabled) + return; + + /* MD initialization. */ + kasan_md_init(); + + /* Now officially enabled. */ + kasan_enabled = true; +} + +static inline const char * +kasan_code_name(uint8_t code) +{ + switch (code) { + case KASAN_GENERIC_REDZONE: + return "GenericRedZone"; + case KASAN_MALLOC_REDZONE: + return "MallocRedZone"; + case KASAN_KMEM_REDZONE: + return "KmemRedZone"; + case KASAN_UMA_FREED: + return "UMAUseAfterFree"; + case KASAN_KSTACK_FREED: + return "KernelStack"; + case 1 ... 7: + return "RedZonePartial"; + case KASAN_STACK_LEFT: + return "StackLeft"; + case KASAN_STACK_MID: + return "StackMiddle"; + case KASAN_STACK_RIGHT: + return "StackRight"; + case KASAN_USE_AFTER_RET: + return "UseAfterRet"; + case KASAN_USE_AFTER_SCOPE: + return "UseAfterScope"; + default: + return "Unknown"; + } +} + +#define REPORT(f, ...) do { \ + if (panic_on_violation) { \ + panic(f, __VA_ARGS__); \ + } else { \ + struct stack st; \ + \ + stack_save(&st); \ + printf(f "\n", __VA_ARGS__); \ + stack_print_ddb(&st); \ + } \ +} while (0) + +static void +kasan_report(unsigned long addr, size_t size, bool write, unsigned long pc, + uint8_t code) +{ + REPORT("ASan: Invalid access, %zu-byte %s at %#lx, %s(%x)", + size, (write ? "write" : "read"), addr, kasan_code_name(code), + code); +} + +static __always_inline void +kasan_shadow_1byte_markvalid(unsigned long addr) +{ + int8_t *byte = (int8_t *)kasan_md_addr_to_shad(addr); + int8_t last = (addr & KASAN_SHADOW_MASK) + 1; + + *byte = last; +} + +static __always_inline void +kasan_shadow_Nbyte_markvalid(const void *addr, size_t size) +{ + size_t i; + + for (i = 0; i < size; i++) { + kasan_shadow_1byte_markvalid((unsigned long)addr + i); + } +} + +static __always_inline void +kasan_shadow_Nbyte_fill(const void *addr, size_t size, uint8_t code) +{ + void *shad; + + if (__predict_false(size == 0)) + return; + if (__predict_false(kasan_md_unsupported((vm_offset_t)addr))) + return; + + KASSERT((vm_offset_t)addr % KASAN_SHADOW_SCALE == 0, + ("%s: invalid address %p", __func__, addr)); + KASSERT(size % KASAN_SHADOW_SCALE == 0, + ("%s: invalid size %zu", __func__, size)); + + shad = (void *)kasan_md_addr_to_shad((uintptr_t)addr); + size = size >> KASAN_SHADOW_SCALE_SHIFT; + + __builtin_memset(shad, code, size); +} + +/* + * In an area of size 'sz_with_redz', mark the 'size' first bytes as valid, + * and the rest as invalid. There are generally two use cases: + * + * o kasan_mark(addr, origsize, size, code), with origsize < size. This marks + * the redzone at the end of the buffer as invalid. If the entire is to be + * marked invalid, origsize will be 0. + * + * o kasan_mark(addr, size, size, 0). This marks the entire buffer as valid. + */ +void +kasan_mark(const void *addr, size_t size, size_t redzsize, uint8_t code) +{ + size_t i, n, redz; + int8_t *shad; + + if ((vm_offset_t)addr >= DMAP_MIN_ADDRESS && + (vm_offset_t)addr < DMAP_MAX_ADDRESS) + return; + + KASSERT((vm_offset_t)addr >= VM_MIN_KERNEL_ADDRESS && + (vm_offset_t)addr < VM_MAX_KERNEL_ADDRESS, + ("%s: invalid address %p", __func__, addr)); + KASSERT((vm_offset_t)addr % KASAN_SHADOW_SCALE == 0, + ("%s: invalid address %p", __func__, addr)); + redz = redzsize - roundup(size, KASAN_SHADOW_SCALE); + KASSERT(redz % KASAN_SHADOW_SCALE == 0, + ("%s: invalid size %zu", __func__, redz)); + shad = (int8_t *)kasan_md_addr_to_shad((uintptr_t)addr); + + /* Chunks of 8 bytes, valid. */ + n = size / KASAN_SHADOW_SCALE; + for (i = 0; i < n; i++) { + *shad++ = 0; + } + + /* Possibly one chunk, mid. */ + if ((size & KASAN_SHADOW_MASK) != 0) { + *shad++ = (size & KASAN_SHADOW_MASK); + } + + /* Chunks of 8 bytes, invalid. */ + n = redz / KASAN_SHADOW_SCALE; + for (i = 0; i < n; i++) { + *shad++ = code; + } +} + +/* -------------------------------------------------------------------------- */ + +#define ADDR_CROSSES_SCALE_BOUNDARY(addr, size) \ + (addr >> KASAN_SHADOW_SCALE_SHIFT) != \ + ((addr + size - 1) >> KASAN_SHADOW_SCALE_SHIFT) + +static __always_inline bool +kasan_shadow_1byte_isvalid(unsigned long addr, uint8_t *code) +{ + int8_t *byte = (int8_t *)kasan_md_addr_to_shad(addr); + int8_t last = (addr & KASAN_SHADOW_MASK) + 1; + + if (__predict_true(*byte == 0 || last <= *byte)) { + return (true); + } + *code = *byte; + return (false); +} + +static __always_inline bool +kasan_shadow_2byte_isvalid(unsigned long addr, uint8_t *code) +{ + int8_t *byte, last; + + if (ADDR_CROSSES_SCALE_BOUNDARY(addr, 2)) { + return (kasan_shadow_1byte_isvalid(addr, code) && + kasan_shadow_1byte_isvalid(addr+1, code)); + } + + byte = (int8_t *)kasan_md_addr_to_shad(addr); + last = ((addr + 1) & KASAN_SHADOW_MASK) + 1; + + if (__predict_true(*byte == 0 || last <= *byte)) { + return (true); + } + *code = *byte; + return (false); +} + +static __always_inline bool +kasan_shadow_4byte_isvalid(unsigned long addr, uint8_t *code) +{ + int8_t *byte, last; + + if (ADDR_CROSSES_SCALE_BOUNDARY(addr, 4)) { + return (kasan_shadow_2byte_isvalid(addr, code) && + kasan_shadow_2byte_isvalid(addr+2, code)); + } + + byte = (int8_t *)kasan_md_addr_to_shad(addr); + last = ((addr + 3) & KASAN_SHADOW_MASK) + 1; + + if (__predict_true(*byte == 0 || last <= *byte)) { + return (true); + } + *code = *byte; + return (false); +} + +static __always_inline bool +kasan_shadow_8byte_isvalid(unsigned long addr, uint8_t *code) +{ + int8_t *byte, last; + + if (ADDR_CROSSES_SCALE_BOUNDARY(addr, 8)) { + return (kasan_shadow_4byte_isvalid(addr, code) && + kasan_shadow_4byte_isvalid(addr+4, code)); + } + + byte = (int8_t *)kasan_md_addr_to_shad(addr); + last = ((addr + 7) & KASAN_SHADOW_MASK) + 1; + + if (__predict_true(*byte == 0 || last <= *byte)) { + return (true); + } + *code = *byte; + return (false); +} + +static __always_inline bool +kasan_shadow_Nbyte_isvalid(unsigned long addr, size_t size, uint8_t *code) +{ + size_t i; + + for (i = 0; i < size; i++) { + if (!kasan_shadow_1byte_isvalid(addr+i, code)) + return (false); + } + + return (true); +} + +static __always_inline void +kasan_shadow_check(unsigned long addr, size_t size, bool write, + unsigned long retaddr) +{ + uint8_t code; + bool valid; + + if (__predict_false(!kasan_enabled)) + return; + if (__predict_false(size == 0)) + return; + if (__predict_false(kasan_md_unsupported(addr))) + return; + if (__predict_false(panicstr != NULL)) + return; + + if (__builtin_constant_p(size)) { + switch (size) { + case 1: + valid = kasan_shadow_1byte_isvalid(addr, &code); + break; + case 2: + valid = kasan_shadow_2byte_isvalid(addr, &code); + break; + case 4: + valid = kasan_shadow_4byte_isvalid(addr, &code); + break; + case 8: + valid = kasan_shadow_8byte_isvalid(addr, &code); + break; + default: + valid = kasan_shadow_Nbyte_isvalid(addr, size, &code); + break; + } + } else { + valid = kasan_shadow_Nbyte_isvalid(addr, size, &code); + } + + if (__predict_false(!valid)) { + kasan_report(addr, size, write, retaddr, code); + } +} + +/* -------------------------------------------------------------------------- */ + +void * +kasan_memcpy(void *dst, const void *src, size_t len) +{ + kasan_shadow_check((unsigned long)src, len, false, __RET_ADDR); + kasan_shadow_check((unsigned long)dst, len, true, __RET_ADDR); + return (__builtin_memcpy(dst, src, len)); +} + +int +kasan_memcmp(const void *b1, const void *b2, size_t len) +{ + kasan_shadow_check((unsigned long)b1, len, false, __RET_ADDR); + kasan_shadow_check((unsigned long)b2, len, false, __RET_ADDR); + return (__builtin_memcmp(b1, b2, len)); +} + +void * +kasan_memset(void *b, int c, size_t len) +{ + kasan_shadow_check((unsigned long)b, len, true, __RET_ADDR); + return (__builtin_memset(b, c, len)); +} + +void * +kasan_memmove(void *dst, const void *src, size_t len) +{ + kasan_shadow_check((unsigned long)src, len, false, __RET_ADDR); + kasan_shadow_check((unsigned long)dst, len, true, __RET_ADDR); + return (__builtin_memmove(dst, src, len)); +} + +size_t +kasan_strlen(const char *str) +{ + const char *s; + + s = str; + while (1) { + kasan_shadow_check((unsigned long)s, 1, false, __RET_ADDR); + if (*s == '\0') + break; + s++; + } + + return (s - str); +} + +char * +kasan_strcpy(char *dst, const char *src) +{ + char *save = dst; + + while (1) { + kasan_shadow_check((unsigned long)src, 1, false, __RET_ADDR); + kasan_shadow_check((unsigned long)dst, 1, true, __RET_ADDR); + *dst = *src; + if (*src == '\0') + break; + src++, dst++; + } + + return save; +} + +int +kasan_strcmp(const char *s1, const char *s2) +{ + while (1) { + kasan_shadow_check((unsigned long)s1, 1, false, __RET_ADDR); + kasan_shadow_check((unsigned long)s2, 1, false, __RET_ADDR); + if (*s1 != *s2) + break; + if (*s1 == '\0') + return 0; + s1++, s2++; + } + + return (*(const unsigned char *)s1 - *(const unsigned char *)s2); +} + +int +kasan_copyin(const void *uaddr, void *kaddr, size_t len) +{ + kasan_shadow_check((unsigned long)kaddr, len, true, __RET_ADDR); + return (copyin(uaddr, kaddr, len)); +} + +int +kasan_copyinstr(const void *uaddr, void *kaddr, size_t len, size_t *done) +{ + kasan_shadow_check((unsigned long)kaddr, len, true, __RET_ADDR); + return (copyinstr(uaddr, kaddr, len, done)); +} + +int +kasan_copyout(const void *kaddr, void *uaddr, size_t len) +{ + kasan_shadow_check((unsigned long)kaddr, len, false, __RET_ADDR); + return (copyout(kaddr, uaddr, len)); +} + +/* -------------------------------------------------------------------------- */ + +#include +#define ATOMIC_SAN_PREFIX kasan +#include + +#define _ASAN_ATOMIC_FUNC_ADD(name, type) \ + void kasan_atomic_add_##name(volatile type *ptr, type val) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + atomic_add_##name(ptr, val); \ + } + +#define ASAN_ATOMIC_FUNC_ADD(name, type) \ + _ASAN_ATOMIC_FUNC_ADD(name, type) \ + _ASAN_ATOMIC_FUNC_ADD(acq_##name, type) \ + _ASAN_ATOMIC_FUNC_ADD(rel_##name, type) + +#define _ASAN_ATOMIC_FUNC_SUBTRACT(name, type) \ + void kasan_atomic_subtract_##name(volatile type *ptr, type val) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + atomic_subtract_##name(ptr, val); \ + } + +#define ASAN_ATOMIC_FUNC_SUBTRACT(name, type) \ + _ASAN_ATOMIC_FUNC_SUBTRACT(name, type) \ + _ASAN_ATOMIC_FUNC_SUBTRACT(acq_##name, type) \ + _ASAN_ATOMIC_FUNC_SUBTRACT(rel_##name, type) + +#define _ASAN_ATOMIC_FUNC_SET(name, type) \ + void kasan_atomic_set_##name(volatile type *ptr, type val) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + atomic_set_##name(ptr, val); \ + } + +#define ASAN_ATOMIC_FUNC_SET(name, type) \ + _ASAN_ATOMIC_FUNC_SET(name, type) \ + _ASAN_ATOMIC_FUNC_SET(acq_##name, type) \ + _ASAN_ATOMIC_FUNC_SET(rel_##name, type) + +#define _ASAN_ATOMIC_FUNC_CLEAR(name, type) \ + void kasan_atomic_clear_##name(volatile type *ptr, type val) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + atomic_clear_##name(ptr, val); \ + } + +#define ASAN_ATOMIC_FUNC_CLEAR(name, type) \ + _ASAN_ATOMIC_FUNC_CLEAR(name, type) \ + _ASAN_ATOMIC_FUNC_CLEAR(acq_##name, type) \ + _ASAN_ATOMIC_FUNC_CLEAR(rel_##name, type) + +#define ASAN_ATOMIC_FUNC_FETCHADD(name, type) \ + type kasan_atomic_fetchadd_##name(volatile type *ptr, type val) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + return (atomic_fetchadd_##name(ptr, val)); \ + } + +#define ASAN_ATOMIC_FUNC_READANDCLEAR(name, type) \ + type kasan_atomic_readandclear_##name(volatile type *ptr) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + return (atomic_readandclear_##name(ptr)); \ + } + +#define ASAN_ATOMIC_FUNC_TESTANDCLEAR(name, type) \ + int kasan_atomic_testandclear_##name(volatile type *ptr, u_int v) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + return (atomic_testandclear_##name(ptr, v)); \ + } + +#define ASAN_ATOMIC_FUNC_TESTANDSET(name, type) \ + int kasan_atomic_testandset_##name(volatile type *ptr, u_int v) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + return (atomic_testandset_##name(ptr, v)); \ + } + +#define ASAN_ATOMIC_FUNC_SWAP(name, type) \ + type kasan_atomic_swap_##name(volatile type *ptr, type val) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + return (atomic_swap_##name(ptr, val)); \ + } + +#define _ASAN_ATOMIC_FUNC_CMPSET(name, type) \ + int kasan_atomic_cmpset_##name(volatile type *ptr, type oval, \ + type nval) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + return (atomic_cmpset_##name(ptr, oval, nval)); \ + } + +#define ASAN_ATOMIC_FUNC_CMPSET(name, type) \ + _ASAN_ATOMIC_FUNC_CMPSET(name, type) \ + _ASAN_ATOMIC_FUNC_CMPSET(acq_##name, type) \ + _ASAN_ATOMIC_FUNC_CMPSET(rel_##name, type) + +#define _ASAN_ATOMIC_FUNC_FCMPSET(name, type) \ + int kasan_atomic_fcmpset_##name(volatile type *ptr, type *oval, \ + type nval) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + return (atomic_fcmpset_##name(ptr, oval, nval)); \ + } + +#define ASAN_ATOMIC_FUNC_FCMPSET(name, type) \ + _ASAN_ATOMIC_FUNC_FCMPSET(name, type) \ + _ASAN_ATOMIC_FUNC_FCMPSET(acq_##name, type) \ + _ASAN_ATOMIC_FUNC_FCMPSET(rel_##name, type) + +#define ASAN_ATOMIC_FUNC_THREAD_FENCE(name) \ + void kasan_atomic_thread_fence_##name(void) \ + { \ + atomic_thread_fence_##name(); \ + } + +#define _ASAN_ATOMIC_FUNC_LOAD(name, type) \ + type kasan_atomic_load_##name(volatile type *ptr) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + return (atomic_load_##name(ptr)); \ + } + +#define ASAN_ATOMIC_FUNC_LOAD(name, type) \ + _ASAN_ATOMIC_FUNC_LOAD(name, type) \ + _ASAN_ATOMIC_FUNC_LOAD(acq_##name, type) + +#define _ASAN_ATOMIC_FUNC_STORE(name, type) \ + void kasan_atomic_store_##name(volatile type *ptr, type val) \ + { \ + kasan_shadow_check((uintptr_t)ptr, sizeof(type), true, \ + __RET_ADDR); \ + atomic_store_##name(ptr, val); \ + } + +#define ASAN_ATOMIC_FUNC_STORE(name, type) \ + _ASAN_ATOMIC_FUNC_STORE(name, type) \ + _ASAN_ATOMIC_FUNC_STORE(rel_##name, type) + +ASAN_ATOMIC_FUNC_ADD(8, uint8_t); +ASAN_ATOMIC_FUNC_ADD(16, uint16_t); +ASAN_ATOMIC_FUNC_ADD(32, uint32_t); +ASAN_ATOMIC_FUNC_ADD(64, uint64_t); +ASAN_ATOMIC_FUNC_ADD(int, u_int); +ASAN_ATOMIC_FUNC_ADD(long, u_long); +ASAN_ATOMIC_FUNC_ADD(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_SUBTRACT(8, uint8_t); +ASAN_ATOMIC_FUNC_SUBTRACT(16, uint16_t); +ASAN_ATOMIC_FUNC_SUBTRACT(32, uint32_t); +ASAN_ATOMIC_FUNC_SUBTRACT(64, uint64_t); +ASAN_ATOMIC_FUNC_SUBTRACT(int, u_int); +ASAN_ATOMIC_FUNC_SUBTRACT(long, u_long); +ASAN_ATOMIC_FUNC_SUBTRACT(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_SET(8, uint8_t); +ASAN_ATOMIC_FUNC_SET(16, uint16_t); +ASAN_ATOMIC_FUNC_SET(32, uint32_t); +ASAN_ATOMIC_FUNC_SET(64, uint64_t); +ASAN_ATOMIC_FUNC_SET(int, u_int); +ASAN_ATOMIC_FUNC_SET(long, u_long); +ASAN_ATOMIC_FUNC_SET(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_CLEAR(8, uint8_t); +ASAN_ATOMIC_FUNC_CLEAR(16, uint16_t); +ASAN_ATOMIC_FUNC_CLEAR(32, uint32_t); +ASAN_ATOMIC_FUNC_CLEAR(64, uint64_t); +ASAN_ATOMIC_FUNC_CLEAR(int, u_int); +ASAN_ATOMIC_FUNC_CLEAR(long, u_long); +ASAN_ATOMIC_FUNC_CLEAR(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_FETCHADD(32, uint32_t); +ASAN_ATOMIC_FUNC_FETCHADD(64, uint64_t); +ASAN_ATOMIC_FUNC_FETCHADD(int, u_int); +ASAN_ATOMIC_FUNC_FETCHADD(long, u_long); + +ASAN_ATOMIC_FUNC_READANDCLEAR(32, uint32_t); +ASAN_ATOMIC_FUNC_READANDCLEAR(64, uint64_t); +ASAN_ATOMIC_FUNC_READANDCLEAR(int, u_int); +ASAN_ATOMIC_FUNC_READANDCLEAR(long, u_long); +ASAN_ATOMIC_FUNC_READANDCLEAR(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_TESTANDCLEAR(32, uint32_t); +ASAN_ATOMIC_FUNC_TESTANDCLEAR(64, uint64_t); +ASAN_ATOMIC_FUNC_TESTANDCLEAR(int, u_int); +ASAN_ATOMIC_FUNC_TESTANDCLEAR(long, u_long); + +ASAN_ATOMIC_FUNC_TESTANDSET(32, uint32_t); +ASAN_ATOMIC_FUNC_TESTANDSET(64, uint64_t); +ASAN_ATOMIC_FUNC_TESTANDSET(int, u_int); +ASAN_ATOMIC_FUNC_TESTANDSET(long, u_long); + +ASAN_ATOMIC_FUNC_SWAP(32, uint32_t); +ASAN_ATOMIC_FUNC_SWAP(64, uint64_t); +ASAN_ATOMIC_FUNC_SWAP(int, u_int); +ASAN_ATOMIC_FUNC_SWAP(long, u_long); +ASAN_ATOMIC_FUNC_SWAP(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_CMPSET(8, uint8_t); +ASAN_ATOMIC_FUNC_CMPSET(16, uint16_t); +ASAN_ATOMIC_FUNC_CMPSET(32, uint32_t); +ASAN_ATOMIC_FUNC_CMPSET(64, uint64_t); +ASAN_ATOMIC_FUNC_CMPSET(int, u_int); +ASAN_ATOMIC_FUNC_CMPSET(long, u_long); +ASAN_ATOMIC_FUNC_CMPSET(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_FCMPSET(8, uint8_t); +ASAN_ATOMIC_FUNC_FCMPSET(16, uint16_t); +ASAN_ATOMIC_FUNC_FCMPSET(32, uint32_t); +ASAN_ATOMIC_FUNC_FCMPSET(64, uint64_t); +ASAN_ATOMIC_FUNC_FCMPSET(int, u_int); +ASAN_ATOMIC_FUNC_FCMPSET(long, u_long); +ASAN_ATOMIC_FUNC_FCMPSET(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_LOAD(8, uint8_t); +ASAN_ATOMIC_FUNC_LOAD(16, uint16_t); +ASAN_ATOMIC_FUNC_LOAD(32, uint32_t); +ASAN_ATOMIC_FUNC_LOAD(64, uint64_t); +ASAN_ATOMIC_FUNC_LOAD(char, u_char); +ASAN_ATOMIC_FUNC_LOAD(short, u_short); +ASAN_ATOMIC_FUNC_LOAD(int, u_int); +ASAN_ATOMIC_FUNC_LOAD(long, u_long); +ASAN_ATOMIC_FUNC_LOAD(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_STORE(8, uint8_t); +ASAN_ATOMIC_FUNC_STORE(16, uint16_t); +ASAN_ATOMIC_FUNC_STORE(32, uint32_t); +ASAN_ATOMIC_FUNC_STORE(64, uint64_t); +ASAN_ATOMIC_FUNC_STORE(char, u_char); +ASAN_ATOMIC_FUNC_STORE(short, u_short); +ASAN_ATOMIC_FUNC_STORE(int, u_int); +ASAN_ATOMIC_FUNC_STORE(long, u_long); +ASAN_ATOMIC_FUNC_STORE(ptr, uintptr_t); + +ASAN_ATOMIC_FUNC_THREAD_FENCE(acq); +ASAN_ATOMIC_FUNC_THREAD_FENCE(rel); +ASAN_ATOMIC_FUNC_THREAD_FENCE(acq_rel); +ASAN_ATOMIC_FUNC_THREAD_FENCE(seq_cst); + +void +kasan_atomic_interrupt_fence(void) +{ +} + +/* -------------------------------------------------------------------------- */ + +#include +#include +#define BUS_SAN_PREFIX kasan +#include + +int +kasan_bus_space_map(bus_space_tag_t tag, bus_addr_t hnd, bus_size_t size, + int flags, bus_space_handle_t *handlep) +{ + return (bus_space_map(tag, hnd, size, flags, handlep)); +} + +void +kasan_bus_space_unmap(bus_space_tag_t tag, bus_space_handle_t hnd, + bus_size_t size) +{ + bus_space_unmap(tag, hnd, size); +} + +int +kasan_bus_space_subregion(bus_space_tag_t tag, bus_space_handle_t hnd, + bus_size_t offset, bus_size_t size, bus_space_handle_t *handlep) +{ + return (bus_space_subregion(tag, hnd, offset, size, handlep)); +} + +void +kasan_bus_space_free(bus_space_tag_t tag, bus_space_handle_t hnd, + bus_size_t size) +{ + bus_space_free(tag, hnd, size); +} + +void +kasan_bus_space_barrier(bus_space_tag_t tag, bus_space_handle_t hnd, + bus_size_t offset, bus_size_t size, int flags) +{ + bus_space_barrier(tag, hnd, offset, size, flags); +} + +#define ASAN_BUS_READ_FUNC(func, width, type) \ + type kasan_bus_space_read##func##_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset) \ + { \ + return (bus_space_read##func##_##width(tag, hnd, \ + offset)); \ + } \ + +#define ASAN_BUS_READ_PTR_FUNC(func, width, type) \ + void kasan_bus_space_read_##func##_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t size, type *buf, \ + bus_size_t count) \ + { \ + kasan_shadow_check((uintptr_t)buf, sizeof(type) * count,\ + false, __RET_ADDR); \ + bus_space_read_##func##_##width(tag, hnd, size, buf, \ + count); \ + } + +ASAN_BUS_READ_FUNC(, 1, uint8_t) +ASAN_BUS_READ_FUNC(_stream, 1, uint8_t) +ASAN_BUS_READ_PTR_FUNC(multi, 1, uint8_t) +ASAN_BUS_READ_PTR_FUNC(multi_stream, 1, uint8_t) +ASAN_BUS_READ_PTR_FUNC(region, 1, uint8_t) +ASAN_BUS_READ_PTR_FUNC(region_stream, 1, uint8_t) + +ASAN_BUS_READ_FUNC(, 2, uint16_t) +ASAN_BUS_READ_FUNC(_stream, 2, uint16_t) +ASAN_BUS_READ_PTR_FUNC(multi, 2, uint16_t) +ASAN_BUS_READ_PTR_FUNC(multi_stream, 2, uint16_t) +ASAN_BUS_READ_PTR_FUNC(region, 2, uint16_t) +ASAN_BUS_READ_PTR_FUNC(region_stream, 2, uint16_t) + +ASAN_BUS_READ_FUNC(, 4, uint32_t) +ASAN_BUS_READ_FUNC(_stream, 4, uint32_t) +ASAN_BUS_READ_PTR_FUNC(multi, 4, uint32_t) +ASAN_BUS_READ_PTR_FUNC(multi_stream, 4, uint32_t) +ASAN_BUS_READ_PTR_FUNC(region, 4, uint32_t) +ASAN_BUS_READ_PTR_FUNC(region_stream, 4, uint32_t) + +ASAN_BUS_READ_FUNC(, 8, uint64_t) + +#define ASAN_BUS_WRITE_FUNC(func, width, type) \ + void kasan_bus_space_write##func##_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type value) \ + { \ + bus_space_write##func##_##width(tag, hnd, offset, value);\ + } \ + +#define ASAN_BUS_WRITE_PTR_FUNC(func, width, type) \ + void kasan_bus_space_write_##func##_##width(bus_space_tag_t tag,\ + bus_space_handle_t hnd, bus_size_t size, const type *buf, \ + bus_size_t count) \ + { \ + kasan_shadow_check((uintptr_t)buf, sizeof(type) * count,\ + true, __RET_ADDR); \ + bus_space_write_##func##_##width(tag, hnd, size, buf, \ + count); \ + } + +ASAN_BUS_WRITE_FUNC(, 1, uint8_t) +ASAN_BUS_WRITE_FUNC(_stream, 1, uint8_t) +ASAN_BUS_WRITE_PTR_FUNC(multi, 1, uint8_t) +ASAN_BUS_WRITE_PTR_FUNC(multi_stream, 1, uint8_t) +ASAN_BUS_WRITE_PTR_FUNC(region, 1, uint8_t) +ASAN_BUS_WRITE_PTR_FUNC(region_stream, 1, uint8_t) + +ASAN_BUS_WRITE_FUNC(, 2, uint16_t) +ASAN_BUS_WRITE_FUNC(_stream, 2, uint16_t) +ASAN_BUS_WRITE_PTR_FUNC(multi, 2, uint16_t) +ASAN_BUS_WRITE_PTR_FUNC(multi_stream, 2, uint16_t) +ASAN_BUS_WRITE_PTR_FUNC(region, 2, uint16_t) +ASAN_BUS_WRITE_PTR_FUNC(region_stream, 2, uint16_t) + +ASAN_BUS_WRITE_FUNC(, 4, uint32_t) +ASAN_BUS_WRITE_FUNC(_stream, 4, uint32_t) +ASAN_BUS_WRITE_PTR_FUNC(multi, 4, uint32_t) +ASAN_BUS_WRITE_PTR_FUNC(multi_stream, 4, uint32_t) +ASAN_BUS_WRITE_PTR_FUNC(region, 4, uint32_t) +ASAN_BUS_WRITE_PTR_FUNC(region_stream, 4, uint32_t) + +ASAN_BUS_WRITE_FUNC(, 8, uint64_t) + +#define ASAN_BUS_SET_FUNC(func, width, type) \ + void kasan_bus_space_set_##func##_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type value, \ + bus_size_t count) \ + { \ + bus_space_set_##func##_##width(tag, hnd, offset, value, \ + count); \ + } + +ASAN_BUS_SET_FUNC(multi, 1, uint8_t) +ASAN_BUS_SET_FUNC(region, 1, uint8_t) +ASAN_BUS_SET_FUNC(multi_stream, 1, uint8_t) +ASAN_BUS_SET_FUNC(region_stream, 1, uint8_t) + +ASAN_BUS_SET_FUNC(multi, 2, uint16_t) +ASAN_BUS_SET_FUNC(region, 2, uint16_t) +ASAN_BUS_SET_FUNC(multi_stream, 2, uint16_t) +ASAN_BUS_SET_FUNC(region_stream, 2, uint16_t) + +ASAN_BUS_SET_FUNC(multi, 4, uint32_t) +ASAN_BUS_SET_FUNC(region, 4, uint32_t) +ASAN_BUS_SET_FUNC(multi_stream, 4, uint32_t) +ASAN_BUS_SET_FUNC(region_stream, 4, uint32_t) + +/* -------------------------------------------------------------------------- */ + +void __asan_register_globals(struct __asan_global *, size_t); +void __asan_unregister_globals(struct __asan_global *, size_t); + +void +__asan_register_globals(struct __asan_global *globals, size_t n) +{ + size_t i; + + for (i = 0; i < n; i++) { + kasan_mark(globals[i].beg, globals[i].size, + globals[i].size_with_redzone, KASAN_GENERIC_REDZONE); + } +} + +void +__asan_unregister_globals(struct __asan_global *globals, size_t n) +{ + /* never called */ +} + +#define ASAN_LOAD_STORE(size) \ + void __asan_load##size(unsigned long); \ + void __asan_load##size(unsigned long addr) \ + { \ + kasan_shadow_check(addr, size, false, __RET_ADDR);\ + } \ + void __asan_load##size##_noabort(unsigned long); \ + void __asan_load##size##_noabort(unsigned long addr) \ + { \ + kasan_shadow_check(addr, size, false, __RET_ADDR);\ + } \ + void __asan_store##size(unsigned long); \ + void __asan_store##size(unsigned long addr) \ + { \ + kasan_shadow_check(addr, size, true, __RET_ADDR);\ + } \ + void __asan_store##size##_noabort(unsigned long); \ + void __asan_store##size##_noabort(unsigned long addr) \ + { \ + kasan_shadow_check(addr, size, true, __RET_ADDR);\ + } + +ASAN_LOAD_STORE(1); +ASAN_LOAD_STORE(2); +ASAN_LOAD_STORE(4); +ASAN_LOAD_STORE(8); +ASAN_LOAD_STORE(16); + +void __asan_loadN(unsigned long, size_t); +void __asan_loadN_noabort(unsigned long, size_t); +void __asan_storeN(unsigned long, size_t); +void __asan_storeN_noabort(unsigned long, size_t); +void __asan_handle_no_return(void); + +void +__asan_loadN(unsigned long addr, size_t size) +{ + kasan_shadow_check(addr, size, false, __RET_ADDR); +} + +void +__asan_loadN_noabort(unsigned long addr, size_t size) +{ + kasan_shadow_check(addr, size, false, __RET_ADDR); +} + +void +__asan_storeN(unsigned long addr, size_t size) +{ + kasan_shadow_check(addr, size, true, __RET_ADDR); +} + +void +__asan_storeN_noabort(unsigned long addr, size_t size) +{ + kasan_shadow_check(addr, size, true, __RET_ADDR); +} + +void +__asan_handle_no_return(void) +{ + /* nothing */ +} + +#define ASAN_SET_SHADOW(byte) \ + void __asan_set_shadow_##byte(void *, size_t); \ + void __asan_set_shadow_##byte(void *addr, size_t size) \ + { \ + __builtin_memset((void *)addr, 0x##byte, size); \ + } + +ASAN_SET_SHADOW(00); +ASAN_SET_SHADOW(f1); +ASAN_SET_SHADOW(f2); +ASAN_SET_SHADOW(f3); +ASAN_SET_SHADOW(f5); +ASAN_SET_SHADOW(f8); + +void __asan_poison_stack_memory(const void *, size_t); +void __asan_unpoison_stack_memory(const void *, size_t); + +void +__asan_poison_stack_memory(const void *addr, size_t size) +{ + size = roundup(size, KASAN_SHADOW_SCALE); + kasan_shadow_Nbyte_fill(addr, size, KASAN_USE_AFTER_SCOPE); +} + +void +__asan_unpoison_stack_memory(const void *addr, size_t size) +{ + kasan_shadow_Nbyte_markvalid(addr, size); +} + +void __asan_alloca_poison(const void *, size_t); +void __asan_allocas_unpoison(const void *, const void *); + +void +__asan_alloca_poison(const void *addr, size_t size) +{ + const void *l, *r; + + KASSERT((vm_offset_t)addr % KASAN_ALLOCA_SCALE_SIZE == 0, + ("%s: invalid address %p", __func__, addr)); + + l = (const uint8_t *)addr - KASAN_ALLOCA_SCALE_SIZE; + r = (const uint8_t *)addr + roundup(size, KASAN_ALLOCA_SCALE_SIZE); + + kasan_shadow_Nbyte_fill(l, KASAN_ALLOCA_SCALE_SIZE, KASAN_STACK_LEFT); + kasan_mark(addr, size, roundup(size, KASAN_ALLOCA_SCALE_SIZE), + KASAN_STACK_MID); + kasan_shadow_Nbyte_fill(r, KASAN_ALLOCA_SCALE_SIZE, KASAN_STACK_RIGHT); +} + +void +__asan_allocas_unpoison(const void *stkbegin, const void *stkend) +{ + size_t size; + + if (__predict_false(!stkbegin)) + return; + if (__predict_false((uintptr_t)stkbegin > (uintptr_t)stkend)) + return; + size = (uintptr_t)stkend - (uintptr_t)stkbegin; + + kasan_shadow_Nbyte_fill(stkbegin, size, 0); +} + +void __asan_poison_memory_region(const void *addr, size_t size); +void __asan_unpoison_memory_region(const void *addr, size_t size); + +void +__asan_poison_memory_region(const void *addr, size_t size) +{ +} + +void +__asan_unpoison_memory_region(const void *addr, size_t size) +{ +} diff --git a/sys/sys/asan.h b/sys/sys/asan.h new file mode 100644 index 000000000000..a8e07b765028 --- /dev/null +++ b/sys/sys/asan.h @@ -0,0 +1,68 @@ +/* $NetBSD: asan.h,v 1.15 2020/09/10 14:10:46 maxv Exp $ */ + +/* + * Copyright (c) 2018-2020 Maxime Villard, m00nbsd.net + * All rights reserved. + * + * This code is part of the KASAN subsystem of the NetBSD kernel. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _SYS_ASAN_H_ +#define _SYS_ASAN_H_ + +#ifdef KASAN +#include + +/* ASAN constants. Part of the compiler ABI. */ +#define KASAN_SHADOW_SCALE 8 +#define KASAN_SHADOW_SCALE_SHIFT 3 + +/* Stack redzone values. Part of the compiler ABI. */ +#define KASAN_STACK_LEFT 0xF1 +#define KASAN_STACK_MID 0xF2 +#define KASAN_STACK_RIGHT 0xF3 +#define KASAN_USE_AFTER_RET 0xF5 +#define KASAN_USE_AFTER_SCOPE 0xF8 + +/* Our redzone values. */ +#define KASAN_GENERIC_REDZONE 0xFA +#define KASAN_MALLOC_REDZONE 0xFB +#define KASAN_KMEM_REDZONE 0xFC +#define KASAN_UMA_FREED 0xFD +#define KASAN_KSTACK_FREED 0xFE + +void kasan_init(void); +void kasan_shadow_map(void *, size_t); + +void kasan_mark(const void *, size_t, size_t, uint8_t); +#else /* KASAN */ +#define kasan_early_init(u) +#define kasan_init() +#define kasan_shadow_map(a, s) +#define kasan_mark(p, s, l, c) +#endif /* !KASAN */ + +#endif /* !_SYS_ASAN_H_ */