Commit Graph

375 Commits

Author SHA1 Message Date
will
ea532c48b8 libkvm: Improve physical address lookup scaling.
Instead of using a hash table to convert physical page addresses to offsets
in the sparse page array, cache the number of bits set for each 4MB chunk of
physical pages.  Upon lookup, find the nearest cached population count, then
add/subtract the number of bits from that point to the page's PTE bit.
Then multiply by page size and add to the sparse page map's base offset.

This replaces O(n) worst-case lookup with O(1) (plus a small number of bits
to scan in the bitmap).  Also, for a 128GB system, a typical kernel core of
about 8GB will now only require ~4.5MB of RAM for this approach instead of
~48MB as with the hash table.

More concretely, /usr/sbin/crashinfo against the same core improves from a
max RSS of 188MB and wall time of 43.72s (33.25 user 2.94 sys) to 135MB and
9.43s (2.58 user 1.47 sys).  Running "thread apply all bt" in kgdb has a
similar RSS improvement, and wall time drops from 4.44s to 1.93s.

Reviewed by:	jhb
Sponsored by:	Backtrace I/O
2016-07-18 01:55:25 +00:00
will
d5ad361553 libkvm: Bounds check (more) PTE indices. 2016-07-18 01:03:39 +00:00
will
5cc6dedd2d libkvm: Put private kvm routines in kvm_private.c. 2016-07-18 01:02:52 +00:00
pfg
9e7434b2c5 lib: minor spelling fixes in comments.
No functional change.
2016-05-01 19:37:33 +00:00
ngie
7dabfb1c6f Don't leak PML4 in _amd64_initvtop(..) if kvm_read2(..) fails
MFC after: 1 week
Reported by: Coverity
CID: 1341474
Sponsored by: EMC / Isilon Storage Division
2016-04-30 09:32:19 +00:00
ngie
e2c5974fd3 Remove a -Wunused-but-set-variable variable -- pa
Reported by: gcc 5.x
Sponsored by: EMC / Isilon Storage Division
2016-04-30 09:31:01 +00:00
ngie
10c1d2d99d Remove kvm_getfiles(3)
This libcall has been broken since (at least) r174989/8.0-RELEASE.

Bump SHLIB_MAJOR for the change

Differential Revision: https://reviews.freebsd.org/D6052
Relnotes: yes
Reviewed by: jhb, markj
Sponsored by: EMC / Isilon Storage Division
2016-04-30 09:21:13 +00:00
ngie
874567745a Fix memory allocation edgecases in kvm_argv(..)
- Don't leak nbufp on realloc failure in kvm_argv
- Catch malloc errors with bufp
- Set buflen last in the "buflen == 0" case to ensure that
  bufp/nbufp is properly reallocated on the next go around

Differential Revision: https://reviews.freebsd.org/D6051
MFC after: 1 week
Reviewed by: jhb, markj
Reported by: cppcheck
Sponsored by: EMC / Isilon Storage Division
2016-04-30 09:13:26 +00:00
ngie
a9f56210b8 Fix up pointer issues with lib/libkvm
In particular,
- avoid dereferencing NULL pointers
- test pointers against NULL, not 0
- test for errout == NULL in the top-level functions (kvm_open, kvm_openfiles,
  kvm_open2, etc)
- Replace a realloc and free on failure with reallocf

Found with: devel/cocchinelle

Differential Revision: https://reviews.freebsd.org/D5954
MFC after: 1 week
Reviewed by: jhb
Sponsored by: EMC / Isilon Storage Division
2016-04-22 18:05:34 +00:00
gjb
1dc4c40e3b MFH
Sponsored by:	The FreeBSD Foundation
2016-04-04 23:55:32 +00:00
jhb
2d884a83b2 Fix parsing of a 1GB page table entry.
Use 'pdpe' to extract the PA of the 1GB page instead of 'pde' (which was
a copy and paste bug from the 2MB page case further down).

CID:		1341467
Reported by:	pfg
2016-03-28 18:41:48 +00:00
gjb
8bfb527a82 MFH
Sponsored by:	The FreeBSD Foundation
2016-02-22 12:28:23 +00:00
skra
10a4549d3c Rename pte.h to pte-v4.h and start including directly either pte-v4.h
or pte-v6.h in files which needs it.

There are quite internal definitions in pte-v4.h and pte-v6.h headers
specific for corresponding pmap implementation. These headers should be
included only in very few files and an intention is to not hide for
which implementation such files are.

Further, sys/arm/arm/elf_trampoline.c is an example of file which
uses armv4 like pmap implementation for both armv4 and armv6 platforms.
This is another reason why pte.h which includes specific header
according to __ARM_ARCH is not created.
2016-02-19 09:23:32 +00:00
skra
aa894ca21b Remove redundant ARM_L2_ADDR_BITS and L2_ADDR_BITS definitions and
replace them by primary ones where needed.
2016-02-18 09:30:04 +00:00
gjb
fef2698edf First pass through library packaging.
Sponsored by:	The FreeBSD Foundation
2016-02-04 21:16:35 +00:00
skra
12fce57c8c Add mmu format info into ARM vmcore.
Fix kvatop translation for 64K pages.

Reviewed by:	jhb
Approved by:	kib (mentor)
Differential Revision:    https://reviews.freebsd.org/D4942
2016-01-15 18:53:06 +00:00
jhb
b499f710a7 Fix building with GCC since PAGE_MASK is signed on i386.
Reviewed by:	ngie
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D4772
2016-01-14 15:51:13 +00:00
jhb
ab35544b83 Add support to libkvm for reading vmcores from other architectures.
- Add a kvaddr_type to represent kernel virtual addresses instead of
  unsigned long.
- Add a struct kvm_nlist which is a stripped down version of struct nlist
  that uses kvaddr_t for n_value.
- Add a kvm_native() routine that returns true if an open kvm descriptor
  is for a native kernel and memory image.
- Add a kvm_open2() function similar to kvm_openfiles().  It drops the
  unused 'swapfile' argument and adds a new function pointer argument for
  a symbol resolving function.  Native kernels still use _fdnlist() from
  libc to resolve symbols if a resolver function is not supplied, but cross
  kernels require a resolver.
- Add a kvm_nlist2() function similar to kvm_nlist() except that it uses
  struct kvm_nlist instead of struct nlist.
- Add a kvm_read2() function similar to kvm_read() except that it uses
  kvaddr_t instead of unsigned long for the kernel virtual address.
- Add a new kvm_arch switch of routines needed by a vmcore backend.
  Each backend is responsible for implementing kvm_read2() for a given
  vmcore format.
- Use libelf to read headers from ELF kernels and cores (except for
  powerpc cores).
- Add internal helper routines for the common page offset hash table used
  by the minidump backends.
- Port all of the existing kvm backends to implement a kvm_arch switch and
  to be cross-friendly by using private constants instead of ones that
  vary by platform (e.g. PAGE_SIZE).  Static assertions are present when
  a given backend is compiled natively to ensure the private constants
  match the real ones.
- Enable all of the existing vmcore backends on all platforms.  This means
  that libkvm on any platform should be able to perform KVA translation
  and read data from a vmcore of any platform.

Tested on:	amd64, i386, sparc64 (marius)
Differential Revision:	https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
jhb
795417cba0 Remove trailing whitespace. 2015-11-26 19:42:10 +00:00
jhb
e670e95fe2 Remove kvm_sparc.c. This is for 32-bit sparc support and was imported
from 4.4BSD.  It has never been used by FreeBSD.
2015-11-26 19:39:15 +00:00
bdrewery
e13d6f8b3f META MODE: Prefer INSTALL=tools/install.sh to lessen the need for xinstall.host.
This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:10:28 +00:00
andrew
8f1f0fd477 Add support to libkvm for reading minidumps on arm64. The kernel side is
missing until it can be cleaned up.

Reviewed by:	jhb
Approved by:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3319
2015-08-20 11:07:51 +00:00
kib
48ccbdea81 The si_status field of the siginfo_t, provided by the waitid(2) and
SIGCHLD signal, should keep full 32 bits of the status passed to the
_exit(2).

Split the combined p_xstat of the struct proc into the separate exit
status p_xexit for normal process exit, and signalled termination
information p_xsig.  Kernel-visible macro KW_EXITCODE() reconstructs
old p_xstat from p_xexit and p_xsig.  p_xexit contains complete status
and copied out into si_status.

Requested by:	Joerg Schilling
Reviewed by:	jilles (previous version), pho
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2015-07-18 09:02:50 +00:00
bapt
594e07bd1b Revert r284417 it is not necessary anymore 2015-06-15 19:28:07 +00:00
bapt
9fb85ece8b Enforce overwritting SHLIBDIR
Since METAMODE has been added, sys.mk loads bsd.mkopt.mk which ends load loading
bsd.own.mk which then defines SHLIBDIR before all the Makefile.inc everywhere.

This makes /lib being populated again.

Reported by:	many
2015-06-15 15:34:20 +00:00
sjg
008d7c831f Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
sjg
75a137820d dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
sjg
65145fa4c8 Merge sync of head 2015-05-27 01:19:58 +00:00
andrew
80e2370786 Add stub functions to build libkvm. There will be added, but we need them
now get the toolchain target building.

Differential Revision:	https://reviews.freebsd.org/D2185
Sponsored by:	The FreeBSD Foundation
2015-04-02 17:30:19 +00:00
sjg
b137080f19 Merge from head@274682 2014-11-19 01:07:58 +00:00
andrew
63825ea043 Allow libkvm to get the kernel va to pa delta without the need for
physaddr. This should allow for a kernel where PHYSADDR and KERNPHYSADDR
are both undefined.

For now libkvm will use the old method of reading physaddr and kernaddr
to allow it to work with old kernels. This could be removed in the future
when enough time has passed.

Differential Revision:	https://reviews.freebsd.org/D939
MFC after:	1 week
2014-10-19 20:23:31 +00:00
adrian
9b44fe556b Update the ULE scheduler + thread and kinfo structs to use int for cpuid
rather than u_char.

To try and play nice with the ABI, the u_char CPU ID values are clamped
at 254.  The new fields now contain the full CPU ID, or -1 for no cpu.

Differential Revision:	D955
Reviewed by:	jhb, kib
Sponsored by:	Norse Corp, Inc.
2014-10-18 19:36:11 +00:00
andrew
055b73bd72 Fix the type of pte_pa as when it is passed into _kvm_pa2off it is either
a uint64_t or a pointer to an off_t. With it being a u_long can result in
incorrect values being returned.
2014-10-16 13:49:54 +00:00
sjg
d7cd1d425c Merge head from 7/28 2014-08-19 06:50:54 +00:00
marcel
9f28abd980 Remove ia64.
This includes:
o   All directories named *ia64*
o   All files named *ia64*
o   All ia64-specific code guarded by __ia64__
o   All ia64-specific makefile logic
o   Mention of ia64 in comments and documentation

This excludes:
o   Everything under contrib/
o   Everything under crypto/
o   sys/xen/interface
o   sys/sys/elf_common.h

Discussed at: BSDcan
2014-07-07 00:27:09 +00:00
sjg
5860f0d106 Updated dependencies 2014-05-16 14:09:51 +00:00
sjg
1a7e48acf1 Updated dependencies 2014-05-10 05:16:28 +00:00
sjg
0c7e03a54c Merge head 2014-04-27 08:13:43 +00:00
andrew
8842d57879 Add a new ARM TARGET_ARCH, armv6hf. This is considered experimental.
This targets the existing ARMv6 and ARMv7 SoCs that contain a VFP unit.
This is an optional coprocessors may not be present in all devices, however
it appears to be in all current SoCs we support.

armv6hf targets the VFP variant of the ARM EABI and our copy of gcc is too
old to support this. Because of this there are a number of WITH/WITHOUT
options that are unsupported and must be left as the default value. The
options and their required value are:
 * WITH_ARM_EABI
 * WITHOUT_GCC
 * WITHOUT_GNUCXX

In addition, without an external toolchain, the following need to be left
as their default:
 * WITH_CLANG
 * WITH_CLANG_IS_CC

As there is a different method of passing float and double values to
functions the ABI is incompatible with existing armv6 binaries. To use
this a full rebuild of world is required. Because no floating point values
are passed into the kernel an armv6 kernel with VFP enabled will work with
an armv6hf userland and vice versa.
2014-03-23 12:49:25 +00:00
glebius
d231df0ee2 Add kvm_getncpus() to obtain mp_ncpus.
Sponsored by:	Nginx, Inc.
2014-02-12 19:22:48 +00:00
emaste
196c663f8d Add bounds check for pteindex / pdeindex 2014-02-12 15:57:38 +00:00
glebius
25d59f119c While it isn't too late and kvm_read_zpcpu() function isn't yet used
outside libkvm(3), change its order of arguments, so that it is the
same as in kvm_read().

Sponsored by:	Nginx, Inc.
2014-02-12 09:41:17 +00:00
jmg
bb95603de9 add support for building a cross-gdb for ARM... This isn't hooked up
to xdev yet as I don't know how to make it work properly...  It also
isn't heavily tested...

Reviewed by:	silence on -arm
2014-02-12 02:08:42 +00:00
marcel
110a8053d6 Handle truncation of the size returned by _kvm_kvatop(). Cores can have
segments larger than INT_MAX.
2014-01-16 06:26:03 +00:00
marcel
f07a13f9eb We don't have to worry about page sizes when working on virtual
cores (i.e. minidumps). Every segment is virtually contiguous.
2014-01-13 19:02:31 +00:00
marcel
29c2e7b72a Re-apply the part of r260022 that was reverted by r260030 with
one significant difference: for LIB32 builds both TARGET_ARCH
and MACHINE_ARCH are defined. TARGET_ARCH confusingly holds the
architecture of the host (e.g. amd64), while MACHINE_ARCH holds
the architecture were trying to build (e.g. i386). With both
set and different, r260022 changed the behaviour to interpret
the condition as building a cross-amd64 libkvm on i386, when
obviously we're trying to build an i386 version on amd64. When
COMPAT_32BIT is defined, we're building LIB32 and ignore the
value of TARGET_ARCH as we did before.
2014-01-13 19:01:14 +00:00
marcel
18470155f6 Revert part of r260022: LIB32 builds were broken by it. 2013-12-29 03:15:15 +00:00
marcel
6891995b94 Add support for virtual cores (aka minidumps). To that end, refactor this
file as follows:
1.  Common ia64-specific support functions have the ia64_ prefix.
2.  Functions that work on physical cores have the phys_ prefix.
3.  Functions that work on virtual cores have the virt_ prefix.

With that:
1.  _kvm_kvatop() has been renamed to phys_kvatop() as it handles
    physical cores only.
2.  The new _kvm_kvatop() is nothing but a wrapper that calls either
    phys_kvatop() or virt_kvatop() by virtue of the kvatop function
    pointer in the vmstate structure.
3.  virt_kvatop() is nothing but a wrapper around virt_addr2off().
4.  virt_addr2off() iterates over the Phdrs to find the segment in
    which the address falls and return the file offset for it.

Now it's up to the kernel to populate the core file appropriately.
2013-12-29 02:31:40 +00:00
marcel
e023c91c02 Allow building a cross libkvm for ia64. 2013-12-28 23:02:48 +00:00
marcel
99153b89d8 Allow building a cross libkvm by setting TARGET_ARCH. The library so
produced will be called libkvm-${ARCH} instead of libkvm. This allows
installing it alongside the native version.
For symbol lookups, use ps_pglobal_lookup() instead of __fdnlist()
when building a cross libkvm. It is assumed that the cross tool that
uses the cross libkvm also provides an implementation for this
proc_services function.

Note that this commit does not change any of the architecture-specific
code for cross-compilation.
2013-12-28 23:01:57 +00:00