Commit Graph

242 Commits

Author SHA1 Message Date
Mark Johnston
9093dd9a66 Implement x86 dtrace_invop_(un)init() in C.
There is no reason for these routines to be written in assembly.  In
the ports of DTrace to other platforms, they are already written in C.
No functional change intended.

MFC after:	1 week
Sponsored by:	Netflix
2019-09-23 15:08:17 +00:00
Mariusz Zaborski
8da024d941 dtrace: 64-bits registers support
The registers in ilumos and FreeBSD have a different number.
In the illumos, last 32-bits register defined is SS an in FreeBSD is GS.
This off-by-one caused the uregs array to returns the wrong 64-bits register
on amd64.

Reviewed by:	markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20363
2019-06-05 22:29:05 +00:00
Justin Hibbits
b2aea1ad8f powerpc/dtrace: Fix fbt function probing for ELFv2
'.' function names exist only in ELFv1.  ELFv2 does away with function
descriptors, and look more like they do on powerpc(32) and most other
platforms, as direct function pointers.  Stop blacklisting regular function
names in ELFv2.

Submitted by:	Brandon Bergren
Differential Revision:	https://reviews.freebsd.org/D20346
2019-05-27 03:18:56 +00:00
Justin Hibbits
d69b94bab0 powerpc/dtrace: Actually fix stack traces
Fix stack unwinding such that requesting N stack frames in lockstat will
actually give you N frames, not anywhere from 0-3 as had been before.

lockstat prints the mutex function instead of the caller as the reported
locker, but the stack frame is detailed enough to find the real caller.

MFC after:	2 weeks
2019-05-17 19:57:08 +00:00
Konstantin Belousov
7c5a46a1bc Remove resolver_qual from DEFINE_IFUNC/DEFINE_UIFUNC macros.
In all practical situations, the resolver visibility is static.

Requested by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Approved by:	so (emaste)
Differential revision:	https://reviews.freebsd.org/D20281
2019-05-16 22:20:54 +00:00
Justin Hibbits
e9aae3496e powerpc/dtrace: Fix dtrace powerpc asm, and simplify stack walking
Fix some execution bugs in the dtrace powerpc asm.  addme pulls in the carry
flag which we don't want, and the result wasn't recorded anyways, so the
following beq to check for exit condition wasn't checking the right
condition.

Simplify the stack walking in dtrace_isa.c, so there's only a single walker
that handles both pc and sp.  This should make it easier to follow, and any
bugfix that may be needed for walking only needs to be made in one place
instead of two now.

MFC after:	2 weeks
2019-04-13 03:32:21 +00:00
Mark Johnston
a4b59d3db6 Use an explicit comparison with VM_GUEST_NO.
Reported by:	jhb
MFC with:	r345359
Sponsored by:	The FreeBSD Foundation
2019-03-21 20:07:50 +00:00
Mark Johnston
e362e590f9 Don't attempt to measure TSC skew when running as a VM guest.
It simply doesn't work in general since VCPUs may migrate between
physical cores.  The approach used to measure skew also doesn't
make much sense in a VM.

PR:		218452
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2019-03-21 02:52:22 +00:00
Mark Johnston
4f1b715c84 Fix a tracepoint lookup race in fasttrap_pid_probe().
fasttrap hooks the userspace breakpoint handler; the hook looks up the
breakpoint address in a hash table of tracepoints.  It is possible for
the tracepoint to be removed by a different thread in between the
breakpoint trap and the hash table lookup, in which case SIGTRAP gets
delivered to the target process.  Fix the problem by adding a
per-process generation counter that gets incremented when a tracepoint
belonging to that process is removed.  Then, when a lookup fails, the
trapping instruction is restarted if the thread's counter doesn't match
that of the process.

Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19273
2019-02-21 22:54:17 +00:00
Mateusz Guzik
8ca79fbd4a dtrace: fix userspace access on boxes with SMAP
dtrace has its own routines which were not updated after SMAP support got
implemented. Use ifunc just like for other routines.

This in particular fixes ustack().

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18542
2018-12-13 20:09:38 +00:00
Justin Hibbits
cfebc0faa7 DTrace/powerpc: Fix FBT return probes
The FBT fuction boundary prober was setting one return probe marker value,
but the dtrace handler was expecting another.  This causes a hang when
tracing return probes.
2018-11-21 16:47:11 +00:00
Justin Hibbits
97a9d3b5c5 powerpc/dtrace: Use explicit bit numbers to mask out PSL_EE
There seems to be a race in CI, such that dtrace_asm.S might be assembled
before the genassym is completed.  This causes a build failure when PSL_EE
doesn't exist, and is read as 0.  Get around this by explicitly specifying
the bits in the mask instead.
2018-10-21 02:08:57 +00:00
John Baldwin
73efa2fbd1 Various fixes for TLB management on RISC-V.
- Remove the arm64-specific cpu_*cache* and cpu_tlb_flush* functions.
  Instead, add RISC-V specific inline functions in cpufunc.h for the
  fence.i and sfence.vma instructions.
- Catch up to changes in the arm64 pmap and remove all the cpu_dcache_*
  calls, pmap_is_current, pmap_l3_valid_cacheable, and PTE_NEXT bits from
  pmap.
- Remove references to the unimplemented riscv_setttb().
- Remove unused cpu_nullop.
- Add a link to the SBI doc to sbi.h.
- Add support for a 4th argument in SBI calls.  It's not documented but
  it seems implied for the asid argument to SBI_REMOVE_SFENCE_VMA_ASID.
- Pass the arguments from sbi_remote_sfence*() to the SEE.  BBL ignores
  them so this is just cosmetic.
- Flush icaches on other CPUs when they resume from kdb in case the
  debugger wrote any breakpoints while the CPUs were paused in the IPI_STOP
  handler.
- Add SMP vs UP versions of pmap_invalidate_* similar to amd64.  The
  UP versions just use simple fences.  The SMP versions use the
  sbi_remove_sfence*() functions to perform TLB shootdowns.  Since we
  don't have a valid pm_active field in the riscv pmap, just IPI all
  CPUs for all invalidations for now.
- Remove an extraneous TLB flush from the end of pmap_bootstrap().
- Don't do a TLB flush when writing new mappings in pmap_enter(), only if
  modifying an existing mapping.  Note that for COW faults a TLB flush is
  only performed after explicitly clearing the old mapping as is done in
  other pmaps.
- Sync the i-cache on all harts before updating the PTE for executable
  mappings in pmap_enter and pmap_enter_quick.  Previously the i-cache was
  only sync'd after updating the PTE in pmap_enter.
- Use sbi_remote_fence() instead of smp_rendezvous in pmap_sync_icache().

Reviewed by:	markj
Approved by:	re (gjb, kib)
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D17414
2018-10-15 18:56:54 +00:00
Ruslan Bukin
378a495661 Add support for 'C'-compressed ISA extension to DTrace FBT provider.
Approved by:	re (kib)
Sponsored by:	DARPA, AFRL
2018-09-03 14:34:09 +00:00
Mark Johnston
a9d49f9e64 Fix the hash table lookup in fbt_destroy().
Reported and tested by:	pho
Approved by:	re (kib)
X-MFC with:	r338359
2018-09-02 17:02:13 +00:00
Mark Johnston
c208cb9923 Allow multiple FBT probes to share a tracepoint.
With GNU ifuncs, multiple FBT probes may correspond to the same
instruction.  fbt_invop() assumed that this could not happen and
would return after the first probe found in the global FBT hash
table, which might not be the one that's enabled.  Fix the problem
on x86 by linking probes that share a tracepoint and having each
linked probe fire when the tracepoint is hit.

PR:		230846
Approved by:	re (gjb)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D16921
2018-08-28 20:21:36 +00:00
Justin Hibbits
7f0df9ac2b dtrace/powerpc: Correct register indices for non-indexed registers in the trapframe
Fix an off-by-one error, LR starts at index 32, not index 33, and the others
follow suit.
2018-07-16 19:47:29 +00:00
Justin Hibbits
a1a990d8a4 Revert r326083, it doesn't behave as expected.
Even though there do appear to be more artificial frames, with 12, stack
traces no longer list at all.  Revert until a better, more stable value can
be determined.
2018-06-03 03:53:11 +00:00
Justin Hibbits
5e91185bb1 Protect dtrace_getpcstack() from a NULL stack pointer in a trap frame
Found when trying to use lockstat on a POWER9, the stack pointer (r1) could
be NULL, and result in a NULL pointer dereference, crashing the kernel.
2018-05-30 03:48:27 +00:00
Mark Johnston
5f05bda607 DTrace aarch64: Avoid calling unwind_frame() in the probe context.
unwind_frame() may be instrumented by FBT, leading to recursion into
dtrace_probe(). Manually inline unwind_frame() as we do with stack
unwinding code for other architectures.

Submitted by:	Domagoj Stolfa
Reviewed by:	manu
MFC after:	1 week
Sponsored by:	DARPA / AFRL
Differential Revision:	https://reviews.freebsd.org/D15359
2018-05-12 15:35:26 +00:00
Mateusz Guzik
9d68f7741f systrace: track it like sdt probes
While here predict false.

Note the code is wrong (regardless of this change). Dereference of the
pointer can race with module unload. A fix would set the probe to a
nop stub instead of NULL.
2018-04-27 15:16:34 +00:00
Mateusz Guzik
7cd794214a dtrace: depessimize dtmalloc when dtrace is active
Each malloc/free was testing dtrace_malloc_enabled and forcing
extra reads from the malloc type struct to see if perhaps a
dtmalloc probe was on.

Treat it like lockstat and sdt: have a global bolean.
2018-04-24 01:06:20 +00:00
Mateusz Guzik
4c5209cb21 lockstat: track lockstat just like sdt probes
In particular flip the frequently tested var to bool.
2018-04-24 01:04:10 +00:00
Mark Johnston
8593136428 Assert that dtrace_probe() doesn't re-enter itself.
This helps catch cases where an instrumented function is called while
in probe context.

Submitted by:	Domagoj Stolfa <domagoj.stolfa@gmail.com>
MFC after:	2 weeks
Sponsored by:	DARPA/AFRL
Differential Revision:	https://reviews.freebsd.org/D14863
2018-04-10 13:47:09 +00:00
John Baldwin
d41e41f9f0 Remove very old and unused signal information codes.
These have been supplanted by the MI signal information codes in
<sys/signal.h> since 7.0.  The FPE_*_TRAP ones were deprecated even
earlier in 1999.

PR:		226579 (exp-run)
Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D14637
2018-03-27 20:57:51 +00:00
Mark Johnston
1de56ac728 Revert part of r331264: disable interrupts before disabling WP.
We might otherwise be preempted, leaving WP disabled while another
thread runs on the CPU.

Reported by:	kib
X-MFC with:	r331264
2018-03-20 21:36:35 +00:00
Mark Johnston
7a79ce2e38 Make use of the KPI added in r331252.
MFC after:	2 weeks
2018-03-20 21:16:26 +00:00
Ed Maste
fc2a8776a2 Rename assym.s to assym.inc
assym is only to be included by other .s files, and should never
actually be assembled by itself.

Reviewed by:	imp, bdrewery (earlier)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14180
2018-03-20 17:58:51 +00:00
Mark Johnston
1aa8a926b8 Unbreak amd64 FBT after r330539.
X-MFC with:	r330539
2018-03-06 15:51:59 +00:00
Mark Johnston
6c7828a280 Avoid CPU migration in dtrace_gethrtime() on x86.
dtrace_gethrtime() may be called outside of probe context, and in
particular, from the DTRACEIOC_BUFSNAP handler.

Disable interrupts rather than using sched_pin() to help ensure that
we don't call any external functions when in probe context.

PR:		218452
MFC after:	1 week
2017-12-18 17:26:24 +00:00
Mark Johnston
0037455148 Don't use pcpu_find() to determine if a CPU ID is valid.
This addresses assertion failures after r326218.

MFC after:	1 week
2017-11-27 18:42:23 +00:00
Justin Hibbits
efa8edd5bb PowerPC has 12 artificial frames for the profiler
It may need to be different between AIM and Book-E, this was tested only on
Book-E (64- and 32-bit)

MFC after:	3 weeks
2017-11-22 01:53:59 +00:00
Mateusz Guzik
5a17c5524f sdt: make all sdt probe sites test one variable
This saves on cache misses at the expense of a slight grow of .text.

Note this is a bandaid for lack of hotpatching.

Discussed with:	markj
2017-10-22 20:22:23 +00:00
Mark Johnston
47f11baaca Use C99 initializers for DTrace provider methods.
This makes the definitions easier to read and more cscope-friendly.

MFC after:	1 week
2017-09-27 17:46:38 +00:00
Kurt Lidl
a8273e4371 Enable dtrace support for mips64 and the ERL kernel config
Turn on the required options in the ERL config file, and ensure
that the fbt module is listed as a dependency for mips in
the modules/dtrace/dtraceall/dtraceall.c file.

PR: 		220346
Reviewed by:	gnn, markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D12227
2017-09-06 03:19:52 +00:00
Ruslan Bukin
ca20f8ec29 o Replace __riscv__ with __riscv
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)

This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.

RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):

__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen

Reviewed by:	ngie
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D11901
2017-08-07 14:09:57 +00:00
Justin Hibbits
880870b41a Follow up r313841 on powerpc
Close a potential race in reading the CPU dtrace flags, where a thread can
start on one CPU, and partway through retrieving the flags be swapped out,
while another thread traps and sets the CPU_DTRACE_NOFAULT.  This could
cause the first thread to return without handling the fault.

Discussed with:	markj@
2017-06-09 20:26:42 +00:00
Mark Johnston
de3a96e3b1 Ensure that profile and tick probes provide a non-zero PC value.
The idle thread may process callouts while reloading the timer in
cpu_activeclock(). In this case, provide a representative value, &cpu_idle,
instead of 0 for args[0] so that the active thread can be more easily
identified from the probe.

This addresses intermittent failures of the profile-n/tst.argtest.d test.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D10651
2017-05-15 21:44:40 +00:00
Justin Hibbits
675cad71e7 Fix stack tracing in dtrace for powerpc
The current method only sort of works, and usually doesn't work reliably.
Also, on Book-E the return address from DEBUG exceptions is not the sentinel
addresses, so it won't exit the loop correctly.

Fix this by better handling trap frames during unwinding, and using the
common trap handler for debug traps, as the code in that segment is
identical between the two.

MFC after:	1 week
2017-05-11 00:23:51 +00:00
Justin Hibbits
679ea09441 Fix the encoded instruction for FBT traps on powerpc
r314370 changed EXC_DTRACE to a different instruction, but neglected to
make the same change to fbt, so dtrace didn't actually pick it up,
resulting in entering KDB instead of trapping for dtrace.

MFC after:	1 week
2017-05-10 03:47:22 +00:00
Justin Hibbits
0440a7f539 Fix check for fbt_excluded() in powerpc
fbt_excluded() returns 1 if the symbol is to be excluded.  Every other
arch has this correct, powerpc was the only broken one

MFC after:	1 week
2017-05-10 03:20:20 +00:00
Mark Johnston
23bff6073b Fix a harmless LOR in dtrace_load().
MFC after:	1 week
2017-05-01 17:01:00 +00:00
Patrick Kelsey
67d955aab4 Corrected misspelled versions of rendezvous.
The MFC will include a compat definition of smp_no_rendevous_barrier()
that calls smp_no_rendezvous_barrier().

Reviewed by:	gnn, kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D10313
2017-04-09 02:00:03 +00:00
Mark Johnston
9fc47d244c Fix a backwards comparison in the code to dump a DTrace debug buffer.
PR:		217739
MFC after:	1 week
2017-03-13 18:43:00 +00:00
Mark Johnston
7174af791e Directly include needed headers rather than relying on pollution.
We get machine/cpu.h via kmem.h -> proc.h -> _vm_domain.h -> seq.h.

Reported by:	Ryan Libby
Sponsored by:	Dell EMC Isilon
X-MFC with:	r313841
2017-02-17 03:27:20 +00:00
Mark Johnston
a11ac730a7 Prevent CPU migration when checking the DTrace nofault flag on x86.
dtrace_trap() consumes page and protection faults triggered by code running
in DTrace probe context. Such faults occur with interrupts disabled and are
detected using a per-CPU flag. Regular faults cause dtrace_trap() to be
called with interrupts enabled, and nothing was ensuring that the flag was
read from the correct CPU. This may result in dtrace_trap() consuming
unrelated page and protection faults when DTrace is enabled, causing the
fault handler to return without actually having handled the fault.

Diagnosed by:	Ryan Libby <rlibby@gmail.com>
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-02-16 23:05:20 +00:00
Mark Johnston
e801af6fba Use PC-relative relocations for USDT probe sites on i386 and amd64.
When recording probe site addresses in the output DOF file, dtrace -G
needs to emit relocations for the .SUNW_dof section in order to obtain
the addresses of functions containing probe sites. DTrace expects the
addresses to be relative to the base address of the final ELF file,
and the amd64 USDT implementation was relying on some unspecified and
incorrect behaviour in the base system GNU ld to achieve this.

This change reimplements the probe site relocation handling to allow
USDT to be used with lld and newer GNU binutils. Specifically, it
makes use of R_X86_64_PC64/R_386_PC32 relocations to obtain the
probe site address relative to the DOF file address, and adds and uses a
new DOF relocation type which computes the final probe site address using
these relative offsets.

Reported by and discussed with:	Rafael Espíndola
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D9374
2017-02-05 02:39:12 +00:00
Mark Johnston
b3b5bfeb22 Sync the x86 dis_tables.c with upstream.
This corresponds to the following illumos issues:

  5755 want support for Intel FMA instrs
  5756 want support for Intel BMI1 instrs
  5757 want support for Intel BMI2 instrs
  5758 want support for Intel AVX2 instrs
  7204 Want broadwell rdseed and adx support
  7208 Want stac/clac disasm support
  7733 Need SHA Instruction dis support
  7756 dis can't handle x86 SSE 3 instructions
  7757 want avx2 disasm tests
  7758 want SSE 4.1 disasm tests

MFC after:	2 weeks
2017-02-03 03:22:47 +00:00
Baptiste Daroussin
b4b4b5304b Revert crap accidentally committed 2017-01-28 16:31:23 +00:00
Baptiste Daroussin
814aaaa7da Revert r312923 a better approach will be taken later 2017-01-28 16:30:14 +00:00