d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
975e1c1ce6
freebsd-dev/lib/libkvm/kvm_private.h

200 lines
7.0 KiB
C
Raw Normal View History

BSD 4.4 Lite Lib Sources
1994-05-27 05:00:24 +00:00
/*-
General further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 3-Clause license. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Special thanks to Wind River for providing access to "The Duke of Highlander" tool: an older (2014) run over FreeBSD tree was useful as a starting point.
2017-11-20 19:49:47 +00:00
* SPDX-License-Identifier: BSD-3-Clause
*
BSD 4.4 Lite Lib Sources
1994-05-27 05:00:24 +00:00
* Copyright (c) 1992, 1993
* The Regents of the University of California. All rights reserved.
*
* This code is derived from software developed by the Computer Systems
* Engineering group at Lawrence Berkeley Laboratory under DARPA contract
* BG 91-66 and contributed to Berkeley.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
Renumber copyright clause 4 Renumber cluase 4 to 3, per what everybody else did when BSD granted them permission to remove clause 3. My insistance on keeping the same numbering for legal reasons is too pedantic, so give up on that point. Submitted by: Jan Schaumann <jschauma@stevens.edu> Pull Request: https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
* 3. Neither the name of the University nor the names of its contributors
BSD 4.4 Lite Lib Sources
1994-05-27 05:00:24 +00:00
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* @(#)kvm_private.h 8.1 (Berkeley) 6/4/93
Use kldsym(2) to lookup symbol values. This avoids the kvm_mkdb juggling and is module aware. Yes, this means that kvm_nlist(3) will find symbols in loaded modules. The emulation of the nlist struct is pretty crude but seems to work well enough for all the users in the tree that I found.
1999-12-27 07:14:58 +00:00
* $FreeBSD$
BSD 4.4 Lite Lib Sources
1994-05-27 05:00:24 +00:00
*/
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
#include <sys/endian.h>
#include <sys/linker_set.h>
#include <gelf.h>
struct kvm_arch {
int (*ka_probe)(kvm_t *);
int (*ka_initvtop)(kvm_t *);
void (*ka_freevtop)(kvm_t *);
int (*ka_kvatop)(kvm_t *, kvaddr_t, off_t *);
int (*ka_native)(kvm_t *);
libkvm: add kvm_walk_pages API. This API allows callers to enumerate all known pages, including any direct map & kernel map virtual addresses, physical addresses, size, offset into the core, & protection configured. For architectures that support direct map addresses, also generate pages for any direct map only addresses that are not associated with kernel map addresses. Fix page size portability issue left behind from previous kvm page table lookup interface. Reviewed by: jhb Sponsored by: Backtrace I/O Differential Revision: https://reviews.freebsd.org/D12279
2017-11-11 23:30:58 +00:00
int (*ka_walk_pages)(kvm_t *, kvm_walk_pages_cb_t *, void *);
Implement kvm_kerndisp This change adds a new libkvm function, kvm_kerndisp(), that can be used to retrieve the kernel displacement, that is the difference between the kernel's base virtual address at run time and the kernel base virtual address specified in the kernel image file. This will be used by kgdb, to properly relocate kernel symbols, when needed. Reviewed by: jhb Differential Revision: https://reviews.freebsd.org/D23285
2020-02-06 13:21:59 +00:00
kssize_t (*ka_kerndisp)(kvm_t *);
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
};
#define KVM_ARCH(ka) DATA_SET(kvm_arch, ka)
BSD 4.4 Lite Lib Sources
1994-05-27 05:00:24 +00:00
struct __kvm {
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
struct kvm_arch *arch;
BSD 4.4 Lite Lib Sources
1994-05-27 05:00:24 +00:00
/*
* a string to be prepended to error messages
* provided for compatibility with sun's interface
* if this value is null, errors are saved in errbuf[]
*/
const char *program;
char *errp; /* XXX this can probably go away */
char errbuf[_POSIX2_LINE_MAX];
#define ISALIVE(kd) ((kd)->vmfd >= 0)
int pmfd; /* physical memory file (or crashdump) */
int vmfd; /* virtual memory file (-1 if crashdump) */
Converted 'vmunix' to 'kernel'.
1994-08-05 09:14:37 +00:00
int nlfd; /* namelist file (e.g., /kernel) */
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
GElf_Ehdr nlehdr; /* ELF file header for namelist file */
int (*resolve_symbol)(const char *, kvaddr_t *);
BSD 4.4 Lite Lib Sources
1994-05-27 05:00:24 +00:00
struct kinfo_proc *procbase;
char *argspc; /* (dynamic) storage for argv strings */
int arglen; /* length of the above */
char **argv; /* (dynamic) storage for argv pointers */
int argc; /* length of above (not actual # present) */
Updates to read the extra indirection in ps_strings. Note that any static executables that depend on this will need to be relinked (ie: do this before 'ps'), but the dynamic linked stuff should be OK (ie: 'w') Obtained from: NetBSD (not much point reinventing the wheel.. :-)
1995-12-09 04:42:51 +00:00
char *argbuf; /* (dynamic) temporary storage */
BSD 4.4 Lite Lib Sources
1994-05-27 05:00:24 +00:00
/*
* Kernel virtual address translation state. This only gets filled
* in for dead kernels; otherwise, the running kernel (i.e. kmem)
* will do the translations for us. It could be big, so we
* only allocate it if necessary.
*/
struct vmstate *vmst;
Re-enable raw dump format support on i386 and amd64 for /dev/fwmem.
2007-06-15 11:35:11 +00:00
int rawdump; /* raw dump format */
libkvm: support access to vmm guest memory, allow writes to fwmem and vmm This change consists of two parts: - allow libkvm to recognize /dev/vmm/* character devices as devices that provide access to the physical memory of a system (similarly to /dev/fwmem*) - allow libkvm to recognize that /dev/vmm/* and /dev/fwmem* devices provide access to the physical memory of live remote systems and, thus, the memory is writable As a result, it should be possible to run commands like $ kgdb -w /path/to/kernel /dev/fwmem0.0 $ kgdb /path/to/kernel /dev/vmm/guest Reviewed by: kib, jhb MFC after: 2 weeks Relnotes: yes Sponsored by: Panzura Differential Revision: https://reviews.freebsd.org/D8679
2016-12-27 10:17:56 +00:00
int writable; /* physical memory is writable */
Make libkvm work on live systems and crashdumps with and without VIMAGE virtualization in the kernel. If we cannot resolve a symbol try to see if we can find it with prefix of the virtualized subsystem, currently only "vnet_entry" by identifying either the vnet of the current process for a live system or the vnet of proc0 (or of dumptid if compiled in a non-default way). The way this is done currently allows us to only touch libkvm but no single application. Once we are going to virtualize more subsystems we will have to review this decision for better scaling. Submitted by: rwatson (initial version of kvm_vnet.c, lots of ideas) Reviewed by: rwatson Approved by: re (kib)
2009-07-23 21:12:21 +00:00
int vnet_initialized; /* vnet fields set up */
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
kvaddr_t vnet_start; /* start of kernel's vnet region */
kvaddr_t vnet_stop; /* stop of kernel's vnet region */
kvaddr_t vnet_current; /* vnet we're working with */
kvaddr_t vnet_base; /* vnet base of current vnet */
A first cut at teaching libkvm how to deal with dynamic per-CPU storage (DPCPU): A new API, kvm_dpcpu_setcpu(3), selects the active CPU for the purposes of DPCPU. Calls to kvm_nlist(3) will automatically translate DPCPU symbols and return a pointer to the current CPU's version of the data. Consumers needing to read the same symbol on several CPUs will invoke a series of setcpu/nlist calls, one per CPU of interest. This addition makes it possible for tools like netstat(1) to query the values of DPCPU variables during crashdump analysis, and is based on similar code handling virtualized global variables. MFC after: 1 week Sponsored by: Juniper Networks, Inc.
2010-03-01 00:27:55 +00:00
/*
* Dynamic per-CPU kernel memory. We translate symbols, on-demand,
* to the data associated with dpcpu_curcpu, set with
* kvm_dpcpu_setcpu().
*/
int dpcpu_initialized; /* dpcpu fields set up */
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
kvaddr_t dpcpu_start; /* start of kernel's dpcpu region */
kvaddr_t dpcpu_stop; /* stop of kernel's dpcpu region */
A first cut at teaching libkvm how to deal with dynamic per-CPU storage (DPCPU): A new API, kvm_dpcpu_setcpu(3), selects the active CPU for the purposes of DPCPU. Calls to kvm_nlist(3) will automatically translate DPCPU symbols and return a pointer to the current CPU's version of the data. Consumers needing to read the same symbol on several CPUs will invoke a series of setcpu/nlist calls, one per CPU of interest. This addition makes it possible for tools like netstat(1) to query the values of DPCPU variables during crashdump analysis, and is based on similar code handling virtualized global variables. MFC after: 1 week Sponsored by: Juniper Networks, Inc.
2010-03-01 00:27:55 +00:00
u_int dpcpu_maxcpus; /* size of base array */
uintptr_t *dpcpu_off; /* base array, indexed by CPU ID */
u_int dpcpu_curcpu; /* CPU we're currently working with */
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
kvaddr_t dpcpu_curoff; /* dpcpu base of current CPU */
libkvm: Improve physical address lookup scaling. Instead of using a hash table to convert physical page addresses to offsets in the sparse page array, cache the number of bits set for each 4MB chunk of physical pages. Upon lookup, find the nearest cached population count, then add/subtract the number of bits from that point to the page's PTE bit. Then multiply by page size and add to the sparse page map's base offset. This replaces O(n) worst-case lookup with O(1) (plus a small number of bits to scan in the bitmap). Also, for a 128GB system, a typical kernel core of about 8GB will now only require ~4.5MB of RAM for this approach instead of ~48MB as with the hash table. More concretely, /usr/sbin/crashinfo against the same core improves from a max RSS of 188MB and wall time of 43.72s (33.25 user 2.94 sys) to 135MB and 9.43s (2.58 user 1.47 sys). Running "thread apply all bt" in kgdb has a similar RSS improvement, and wall time drops from 4.44s to 1.93s. Reviewed by: jhb Sponsored by: Backtrace I/O
2016-07-18 01:55:25 +00:00
/* Page table lookup structures. */
uint64_t *pt_map;
size_t pt_map_size;
Sparsify the vm_page_dump bitmap On Ampere Altra systems, the sparse population of RAM within the physical address space causes the vm_page_dump bitmap to be much larger than necessary, increasing the size from ~8 Mib to > 2 Gib (and overflowing `int` for the size). Changing the page dump bitmap also changes the minidump file format, so changes are also necessary in libkvm. Reviewed by: jhb Approved by: scottl (implicit) MFC after: 1 week Sponsored by: Ampere Computing, Inc. Differential Revision: https://reviews.freebsd.org/D26131
2020-09-21 22:21:59 +00:00
uint64_t *dump_avail; /* actually word sized */
size_t dump_avail_size;
libkvm: Improve physical address lookup scaling. Instead of using a hash table to convert physical page addresses to offsets in the sparse page array, cache the number of bits set for each 4MB chunk of physical pages. Upon lookup, find the nearest cached population count, then add/subtract the number of bits from that point to the page's PTE bit. Then multiply by page size and add to the sparse page map's base offset. This replaces O(n) worst-case lookup with O(1) (plus a small number of bits to scan in the bitmap). Also, for a 128GB system, a typical kernel core of about 8GB will now only require ~4.5MB of RAM for this approach instead of ~48MB as with the hash table. More concretely, /usr/sbin/crashinfo against the same core improves from a max RSS of 188MB and wall time of 43.72s (33.25 user 2.94 sys) to 135MB and 9.43s (2.58 user 1.47 sys). Running "thread apply all bt" in kgdb has a similar RSS improvement, and wall time drops from 4.44s to 1.93s. Reviewed by: jhb Sponsored by: Backtrace I/O
2016-07-18 01:55:25 +00:00
off_t pt_sparse_off;
uint64_t pt_sparse_size;
uint32_t *pt_popcounts;
unsigned int pt_page_size;
unsigned int pt_word_size;
libkvm: add kvm_walk_pages API. This API allows callers to enumerate all known pages, including any direct map & kernel map virtual addresses, physical addresses, size, offset into the core, & protection configured. For architectures that support direct map addresses, also generate pages for any direct map only addresses that are not associated with kernel map addresses. Fix page size portability issue left behind from previous kvm page table lookup interface. Reviewed by: jhb Sponsored by: Backtrace I/O Differential Revision: https://reviews.freebsd.org/D12279
2017-11-11 23:30:58 +00:00
/* Page & sparse map structures. */
void *page_map;
uint32_t page_map_size;
off_t page_map_off;
void *sparse_map;
};
struct kvm_bitmap {
uint8_t *map;
u_long size;
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
};
libkvm: Improve physical address lookup scaling. Instead of using a hash table to convert physical page addresses to offsets in the sparse page array, cache the number of bits set for each 4MB chunk of physical pages. Upon lookup, find the nearest cached population count, then add/subtract the number of bits from that point to the page's PTE bit. Then multiply by page size and add to the sparse page map's base offset. This replaces O(n) worst-case lookup with O(1) (plus a small number of bits to scan in the bitmap). Also, for a 128GB system, a typical kernel core of about 8GB will now only require ~4.5MB of RAM for this approach instead of ~48MB as with the hash table. More concretely, /usr/sbin/crashinfo against the same core improves from a max RSS of 188MB and wall time of 43.72s (33.25 user 2.94 sys) to 135MB and 9.43s (2.58 user 1.47 sys). Running "thread apply all bt" in kgdb has a similar RSS improvement, and wall time drops from 4.44s to 1.93s. Reviewed by: jhb Sponsored by: Backtrace I/O
2016-07-18 01:55:25 +00:00
/* Page table lookup constants. */
#define POPCOUNT_BITS 1024
#define BITS_IN(v) (sizeof(v) * NBBY)
#define POPCOUNTS_IN(v) (POPCOUNT_BITS / BITS_IN(v))
BSD 4.4 Lite Lib Sources
1994-05-27 05:00:24 +00:00
/*
* Functions used internally by kvm, but across kvm modules.
*/
[PowerPC64LE] libkvm powerpc64le support. * Add missing _kvm16toh() function. * Teach libkvm about powerpc64le. Sponsored by: Tag1 Consulting, Inc.
2020-09-23 02:11:24 +00:00
static inline uint16_t
_kvm16toh(kvm_t *kd, uint16_t val)
{
if (kd->nlehdr.e_ident[EI_DATA] == ELFDATA2LSB)
return (le16toh(val));
else
return (be16toh(val));
}
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
static inline uint32_t
_kvm32toh(kvm_t *kd, uint32_t val)
{
if (kd->nlehdr.e_ident[EI_DATA] == ELFDATA2LSB)
return (le32toh(val));
else
return (be32toh(val));
}
static inline uint64_t
_kvm64toh(kvm_t *kd, uint64_t val)
{
if (kd->nlehdr.e_ident[EI_DATA] == ELFDATA2LSB)
return (le64toh(val));
else
return (be64toh(val));
}
Sparsify the vm_page_dump bitmap On Ampere Altra systems, the sparse population of RAM within the physical address space causes the vm_page_dump bitmap to be much larger than necessary, increasing the size from ~8 Mib to > 2 Gib (and overflowing `int` for the size). Changing the page dump bitmap also changes the minidump file format, so changes are also necessary in libkvm. Reviewed by: jhb Approved by: scottl (implicit) MFC after: 1 week Sponsored by: Ampere Computing, Inc. Differential Revision: https://reviews.freebsd.org/D26131
2020-09-21 22:21:59 +00:00
uint64_t _kvm_pa_bit_id(kvm_t *kd, uint64_t pa, unsigned int page_size);
uint64_t _kvm_bit_id_pa(kvm_t *kd, uint64_t bit_id, unsigned int page_size);
#define _KVM_PA_INVALID ULONG_MAX
#define _KVM_BIT_ID_INVALID ULONG_MAX
libkvm: add kvm_walk_pages API. This API allows callers to enumerate all known pages, including any direct map & kernel map virtual addresses, physical addresses, size, offset into the core, & protection configured. For architectures that support direct map addresses, also generate pages for any direct map only addresses that are not associated with kernel map addresses. Fix page size portability issue left behind from previous kvm page table lookup interface. Reviewed by: jhb Sponsored by: Backtrace I/O Differential Revision: https://reviews.freebsd.org/D12279
2017-11-11 23:30:58 +00:00
int _kvm_bitmap_init(struct kvm_bitmap *, u_long, u_long *);
Sparsify the vm_page_dump bitmap On Ampere Altra systems, the sparse population of RAM within the physical address space causes the vm_page_dump bitmap to be much larger than necessary, increasing the size from ~8 Mib to > 2 Gib (and overflowing `int` for the size). Changing the page dump bitmap also changes the minidump file format, so changes are also necessary in libkvm. Reviewed by: jhb Approved by: scottl (implicit) MFC after: 1 week Sponsored by: Ampere Computing, Inc. Differential Revision: https://reviews.freebsd.org/D26131
2020-09-21 22:21:59 +00:00
void _kvm_bitmap_set(struct kvm_bitmap *, u_long);
libkvm: add kvm_walk_pages API. This API allows callers to enumerate all known pages, including any direct map & kernel map virtual addresses, physical addresses, size, offset into the core, & protection configured. For architectures that support direct map addresses, also generate pages for any direct map only addresses that are not associated with kernel map addresses. Fix page size portability issue left behind from previous kvm page table lookup interface. Reviewed by: jhb Sponsored by: Backtrace I/O Differential Revision: https://reviews.freebsd.org/D12279
2017-11-11 23:30:58 +00:00
int _kvm_bitmap_next(struct kvm_bitmap *, u_long *);
void _kvm_bitmap_deinit(struct kvm_bitmap *);
Remove multi-line __P() usage.
2002-03-22 09:22:15 +00:00
void _kvm_err(kvm_t *kd, const char *program, const char *fmt, ...)
__printflike(3, 4);
Remove __P() usage.
2002-03-21 23:54:04 +00:00
void _kvm_freeprocs(kvm_t *kd);
void *_kvm_malloc(kvm_t *kd, size_t);
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
int _kvm_nlist(kvm_t *, struct kvm_nlist *, int);
Remove __P() usage.
2002-03-21 23:54:04 +00:00
void *_kvm_realloc(kvm_t *kd, void *, size_t);
Remove multi-line __P() usage.
2002-03-22 09:22:15 +00:00
void _kvm_syserr (kvm_t *kd, const char *program, const char *fmt, ...)
__printflike(3, 4);
Make libkvm work on live systems and crashdumps with and without VIMAGE virtualization in the kernel. If we cannot resolve a symbol try to see if we can find it with prefix of the virtualized subsystem, currently only "vnet_entry" by identifying either the vnet of the current process for a live system or the vnet of proc0 (or of dumptid if compiled in a non-default way). The way this is done currently allows us to only touch libkvm but no single application. Once we are going to virtualize more subsystems we will have to review this decision for better scaling. Submitted by: rwatson (initial version of kvm_vnet.c, lots of ideas) Reviewed by: rwatson Approved by: re (kib)
2009-07-23 21:12:21 +00:00
int _kvm_vnet_selectpid(kvm_t *, pid_t);
int _kvm_vnet_initialized(kvm_t *, int);
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
kvaddr_t _kvm_vnet_validaddr(kvm_t *, kvaddr_t);
A first cut at teaching libkvm how to deal with dynamic per-CPU storage (DPCPU): A new API, kvm_dpcpu_setcpu(3), selects the active CPU for the purposes of DPCPU. Calls to kvm_nlist(3) will automatically translate DPCPU symbols and return a pointer to the current CPU's version of the data. Consumers needing to read the same symbol on several CPUs will invoke a series of setcpu/nlist calls, one per CPU of interest. This addition makes it possible for tools like netstat(1) to query the values of DPCPU variables during crashdump analysis, and is based on similar code handling virtualized global variables. MFC after: 1 week Sponsored by: Juniper Networks, Inc.
2010-03-01 00:27:55 +00:00
int _kvm_dpcpu_initialized(kvm_t *, int);
Add support to libkvm for reading vmcores from other architectures. - Add a kvaddr_type to represent kernel virtual addresses instead of unsigned long. - Add a struct kvm_nlist which is a stripped down version of struct nlist that uses kvaddr_t for n_value. - Add a kvm_native() routine that returns true if an open kvm descriptor is for a native kernel and memory image. - Add a kvm_open2() function similar to kvm_openfiles(). It drops the unused 'swapfile' argument and adds a new function pointer argument for a symbol resolving function. Native kernels still use _fdnlist() from libc to resolve symbols if a resolver function is not supplied, but cross kernels require a resolver. - Add a kvm_nlist2() function similar to kvm_nlist() except that it uses struct kvm_nlist instead of struct nlist. - Add a kvm_read2() function similar to kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. - Add a new kvm_arch switch of routines needed by a vmcore backend. Each backend is responsible for implementing kvm_read2() for a given vmcore format. - Use libelf to read headers from ELF kernels and cores (except for powerpc cores). - Add internal helper routines for the common page offset hash table used by the minidump backends. - Port all of the existing kvm backends to implement a kvm_arch switch and to be cross-friendly by using private constants instead of ones that vary by platform (e.g. PAGE_SIZE). Static assertions are present when a given backend is compiled natively to ensure the private constants match the real ones. - Enable all of the existing vmcore backends on all platforms. This means that libkvm on any platform should be able to perform KVA translation and read data from a vmcore of any platform. Tested on: amd64, i386, sparc64 (marius) Differential Revision: https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
kvaddr_t _kvm_dpcpu_validaddr(kvm_t *, kvaddr_t);
int _kvm_probe_elf_kernel(kvm_t *, int, int);
int _kvm_is_minidump(kvm_t *);
int _kvm_read_core_phdrs(kvm_t *, size_t *, GElf_Phdr **);
Sparsify the vm_page_dump bitmap On Ampere Altra systems, the sparse population of RAM within the physical address space causes the vm_page_dump bitmap to be much larger than necessary, increasing the size from ~8 Mib to > 2 Gib (and overflowing `int` for the size). Changing the page dump bitmap also changes the minidump file format, so changes are also necessary in libkvm. Reviewed by: jhb Approved by: scottl (implicit) MFC after: 1 week Sponsored by: Ampere Computing, Inc. Differential Revision: https://reviews.freebsd.org/D26131
2020-09-21 22:21:59 +00:00
int _kvm_pt_init(kvm_t *, size_t, off_t, size_t, off_t, off_t, int, int);
libkvm: add kvm_walk_pages API. This API allows callers to enumerate all known pages, including any direct map & kernel map virtual addresses, physical addresses, size, offset into the core, & protection configured. For architectures that support direct map addresses, also generate pages for any direct map only addresses that are not associated with kernel map addresses. Fix page size portability issue left behind from previous kvm page table lookup interface. Reviewed by: jhb Sponsored by: Backtrace I/O Differential Revision: https://reviews.freebsd.org/D12279
2017-11-11 23:30:58 +00:00
off_t _kvm_pt_find(kvm_t *, uint64_t, unsigned int);
int _kvm_visit_cb(kvm_t *, kvm_walk_pages_cb_t *, void *, u_long,
u_long, u_long, vm_prot_t, size_t, unsigned int);
int _kvm_pmap_init(kvm_t *, uint32_t, off_t);
void * _kvm_pmap_get(kvm_t *, u_long, size_t);
void * _kvm_map_get(kvm_t *, u_long, unsigned int);
Reference in New Issue Copy Permalink