When examining existing processes pmcstat fails to
correctly determine the locations of executable sections
of the process due to a miscalculated virtual load address.
This does not affect the newly launched processes as the
same value passed as a "start address" to the pmcstat_image_link()
thus nullifying the effect of it. The issue manifests itself
in processes not being reported in the pmcstat(8) output and
"dubious frames" being reported.
Fix it for now by ignoring all the sections except the executable
one. This won't fix the issue for objects with multiple
executable sections but helps in majority of real world usecases.
The real solution would be to modify the MAP-IN event to include
the appropriate load address so pmcstat(8) won't have to manually
parse object files to try to determine it.
PR: 198147, 198148
Reviewed by: jhb, rpaulo
MFC after: 2 weeks
In one case generating callgraph output from a 24MB system-wide sampling
data file took 17.4 seconds on average. Profiling showed pmcstat
spending a lot of time in strcmp, due to hash collisions.
Replacing the XOR-only hash with FNV-1a reduces the run time for my
test by 40%.
'-m <file>' spits out the given stream into <file> (eg, /dev/stdout).
However, it only resolves the first symbol; it doesn't parse the entire
callgraph. If it fails to lookup then it doesn't print anything.
'-a' instead does a symbol and file:line lookup for each address in each
callgraph and will happily print the address itself with no lookup
information if it couldn't look things up.
This makes it much easier to pull out individual records from a
pmc data file and look at the callgraph information without having to
hand-decode the addresses.
Sponsored by: Netflix, Inc.
In addition to adding `static' where possible:
- bin/date: Move `retval' into extern.h to make it visible to date.c.
- bin/ed: Move globally used variables into ed.h.
- sbin/camcontrol: Move `verbose' into camcontrol.h and fix shadow warnings.
- usr.bin/calendar: Remove unneeded variables.
- usr.bin/chat: Make `line' local instead of global.
- usr.bin/elfdump: Comment out unneeded function.
- usr.bin/rlogin: Use _Noreturn instead of __dead2.
- usr.bin/tset: Pull `Ospeed' into extern.h.
- usr.sbin/mfiutil: Put global variables in mfiutil.h.
- usr.sbin/pkg: Remove unused `os_corres'.
- usr.sbin/quotaon, usr.sbin/repquota: Remove unused `qfname'.
New kernel events can be added at various location for sampling or counting.
This will for example allow easy system profiling whatever the processor is
with known tools like pmcstat(8).
Simultaneous usage of software PMC and hardware PMC is possible, for example
looking at the lock acquire failure, page fault while sampling on
instructions.
Sponsored by: NETASQ
MFC after: 1 month
In case of multiple level of inlining all the locations are flattened.
Require recent binutils/addr2line (head works or binutils from ports
with the right $PATH order).
- Multiple fixes in the calltree output (recursion case, ...)
- Fix the calltree top view that previously hide some shared nodes.
Tested with Kcachegrind(kdesdk4)/qcachegrind(head).
Sponsored by: NETASQ
As the underlying block is 4KB if the PMC throughput is low the measurement
will be reported on the next tick. pmcstat(8) use the modified flush API to
reclaim current buffer before displaying next top.
MFC after: 1 month
ints. That fixes a first bug where pmcstat wasn't using the old
cpumask_t interface and now also brings the full support for more
than 32 cpus.
While here, make the functions pmcstat_clone_event_descriptor() and
pmcstat_get_cpumask() private to pmcstat.
The problem of assuming cpu dense masks still persists and should be
eventually fixed, as reported by avg.
Tested by: pluknet
Reviewed by: gnn
Approved by: re (kib)
will be spread as small value and then filtered by the threshold.
As a first step solution display the number of event that cannot
be resolved as a valid function location.
MFC after: 1week
This will allow top monitoring using socket/ssh tunnelling
of system without local symbols.
client:
pmcstat -R <ip>:<port> -T -r <symbolspath>
monitored device:
pmcstat -Sinstructions -O <ip>:<port>
- Move the file read in the event loop
- Initialize and clean log in all cases
- Preserve global stats value during top refresh
- Fix the rtld/line resolver that ignore '-r' prefix
- Support socket for '-R' (server mode)
- Display the statistics when exiting top mode
pmc_flush_logfile is now non-blocking and just ask the kernel
to shutdown the file. From that point, no more data is
accepted by the log thread and when the last buffer is flushed
the file is closed.
This will remove a deadlock between pmcstat asking for
flush while it cannot flush the pipe itself.
MFC after: 3 days
- Kcachegrind (calltree) support with assembly/source
code mapping and call count estimator (-F).
- Top mode for calltree and callgraph plugin (-T).
MFC after: 1 month
Fix some wrong usages.
Note: this does not affect generated binaries as this argument is not used.
PR: 137213
Submitted by: Eygene Ryabinkin (initial version)
MFC after: 1 month
This option prints the list of sampled PCs along with the function name,
the start and end addresses of this where their live within.
Reviewed by: jkoshy
Tested by: gnn
Sponsored by: Nokia
- Print a warning if the version number recorded in the log is not what the
tool expects.
- Print a tidier error message when an unrecognized event is encountered
in the log.
- Don't print a spurious 'Unknown error: 0' when exiting after a parse
error.
takes a host:port specification.
- Update the manual page and add an example showing how log
over the network using pmcstat(8) and nc(1). Document the
current inability to process logs in cross-platform manner.
- Have pmcstat_open_log() call err(3) directly in case
of an error; this simplifies error handling in its caller.
MFC after: 1 week
Kernel changes:
Inform hwpmc of executable objects brought into the system by
kldload() and mmap(), and of their removal by kldunload() and
munmap(). A helper function linker_hwpmc_list_objects() has been
added to "sys/kern/kern_linker.c" and is used by hwpmc to retrieve
the list of currently loaded kernel modules.
The unused `MAPPINGCHANGE' event has been deprecated in favour
of separate `MAP_IN' and `MAP_OUT' events; this change reduces
space wastage in the log.
Bump the hwpmc's ABI version to "2.0.00". Teach hwpmc(4) to
handle the map change callbacks.
Change the default per-cpu sample buffer size to hold
32 samples (up from 16).
Increment __FreeBSD_version.
libpmc(3) changes:
Update libpmc(3) to deal with the new events in the log file; bring
the pmclog(3) manual page in sync with the code.
pmcstat(8) changes:
Introduce new options to pmcstat(8): "-r" (root fs path), "-M"
(mapfile name), "-q"/"-v" (verbosity control). Option "-k" now
takes a kernel directory as its argument but will also work with
the older invocation syntax.
Rework string handling in pmcstat(8) to use an opaque type for
interned strings. Clean up ELF parsing code and add support for
tracking dynamic object mappings reported by a v2.0.00 hwpmc(4).
Report statistics at the end of a log conversion run depending
on the requested verbosity level.
Reviewed by: jhb, dds (kernel parts of an earlier patch)
Tested by: gallatin (earlier patch)
linked process was exec()'ed, and map samples in that address range
into the appropriate buckets in the runtime loader's profile.
Improve a few comments.
MFC after: 3 days
- pmcstat(8) gprof output mode fixes:
lib/libpmc/pmclog.{c,h}, sys/sys/pmclog.h:
+ Add a 'is_usermode' field to the PMCLOG_PCSAMPLE event
+ Add an 'entryaddr' field to the PMCLOG_PROCEXEC event,
so that pmcstat(8) can determine where the runtime loader
/libexec/ld-elf.so.1 is getting loaded.
sys/kern/kern_exec.c:
+ Use a local struct to group the entry address of the image being
exec()'ed and the process credential changed flag to the exec
handling hook inside hwpmc(4).
usr.sbin/pmcstat/*:
+ Support "-k kernelpath", "-D sampledir".
+ Implement the ELF bits of 'gmon.out' profile generation in a new
file "pmcstat_log.c". Move all log related functions to this
file.
+ Move local definitions and prototypes to "pmcstat.h"
- Other bug fixes:
+ lib/libpmc/pmclog.c: correctly handle EOF in pmclog_read().
+ sys/dev/hwpmc_mod.c: unconditionally log a PROCEXIT event to all
attached PMCs when a process exits.
+ sys/sys/pmc.h: correct a function prototype.
+ Improve usage checks in pmcstat(8).
Approved by: re (blanket hwpmc)
