Commit Graph

17524 Commits

Author SHA1 Message Date
araujo
077431a537 Fix style(9) space vs tab.
Reviewed by:	jhb
MFC after:	3 weeks.
Sponsored by:	iXsystems Inc.
Differential Revision:	https://reviews.freebsd.org/D15768
2018-06-14 01:34:53 +00:00
araujo
46169e10ff While I was investigating CID 1194192 related with a resource leak on mrp memory
allocation, I could identify that actually we use this pointer on pci_emul.c as
well as on vga.c source file.

I have reworked the logic here to make it more readable and also add a warn to
explicit show the function where the memory allocation error could happen,
also sort headers.

Also CID 1194192 was marked as "Intentional".

Obtained from:	TrueOS
MFC after:	4 weeks.
Sponsored by:	iXsystems Inc.
2018-06-13 11:49:34 +00:00
eadler
7021b99bc6 pmc: remove trailing whitespace
Reported by:	swills
2018-06-13 09:17:04 +00:00
imp
3ecb2b8fca Use "$@" instead of $* to cope with parameters that have spaces in
them. "$@" preserves the args with spaces properly.

Differential Revision: https://reviews.freebsd.org/D15784
2018-06-13 06:11:04 +00:00
araujo
c4f7676726 When this code was introduced at r300829 the author forgot to add
the BSD license header that is the same as in its C header file.

MFC after:	4 weeks.
Sponsored by:	iXsystems Inc.
2018-06-13 04:00:21 +00:00
araujo
6c496fe989 style(9) remove unnecessary blank tabs.
Obtained from:	TrueOS
MFC after:	4 weeks.
Sponsored by:	iXsystems Inc.
2018-06-13 03:35:24 +00:00
araujo
7aa635aaa7 Add SPDX tags to bhyve(8).
Discussed with:	rgrimes, pfg and mav.
Obtained from:	TrueOS
MFC after:	4 weeks.
Sponsored by:	iXsystems Inc.
2018-06-13 03:22:08 +00:00
sbruno
8520e7ee52 cpucontrol:
- fix debugging for family on AMD cpus and add useful debugging for
  which file is being selected for update.

Reviewed by:	cem
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15574
2018-06-12 18:58:56 +00:00
rlibby
68ad8264a2 pmc gcc fixups
Fix the build of lib/libpmc and usr.sbin/pmc for gcc on amd64.

Reviewed by:    mmacy
Sponsored by:   Dell EMC Isilon
Differential Revision:  https://reviews.freebsd.org/D15723
2018-06-11 16:09:54 +00:00
araujo
7ea85b739f More style(9) fixes, space vs tab. 2018-06-11 15:18:31 +00:00
pfg
31bc8145b0 style(9): Fix tabs after #define.
No functional change intended.
2018-06-11 14:45:34 +00:00
araujo
64730b7ec2 - Add bhyve virtio-scsi storage backend support.
Example of configuration:
ctl.conf:
portal-group pg0 {
        discovery-auth-group no-authentication
        listen 0.0.0.0
        listen [::]
}

target iqn.2012-06.com.example:target0 {
        auth-group no-authentication
        portal-group pg0
        port ioctl/5/3

        lun 0 {
                path /z/test.img
                size 8G
        }
        lun 1 {
                path /z/test1.img
                size 8G
        }
}

bhyve <...> -s 4,virtio-scsi,/dev/cam/ctl5.3,iid=3 <VM_NAME>

From inside guest:
root@:~ # zpool status test
  pool: test
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        test        ONLINE       0     0     0
          da0       ONLINE       0     0     0
          da1       ONLINE       0     0     0

dmesg:
da0 at vtscsi0 bus 0 scbus0 target 0 lun 0
da0: <FREEBSD CTLDISK 0001> Fixed Direct Access SPC-5 SCSI device
da0: Serial Number MYSERIAL0000
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 8192MB (16777216 512 byte sectors)
da1 at vtscsi0 bus 0 scbus0 target 0 lun 1
da1: <FREEBSD CTLDISK 0001> Fixed Direct Access SPC-5 SCSI device
da1: Serial Number MYSERIAL0001
da1: 300.000MB/s transfers
da1: Command Queueing enabled
da1: 8192MB (16777216 512 byte sectors)

Discussed with:		grehan
Reviewed by:		mav
Obtained from:		TrueOS
Relnotes:		Yes
Sponsored by:		iXsystems Inc.
Tested with:		FreeBSD HEAD, Fedora 28 (Workstation) and
			Ubuntu 18.04.
Differential Revision:  https://reviews.freebsd.org/D15276
2018-06-11 02:09:20 +00:00
imp
88e87ea5ab Remove old, dead compat code.
We no longer need to od these things conditionally, and the fallbacks
are to 4.2BSD era defaults, which nobody uses anymore. Vixie cron has
diverged from upstream anyway in our tree, and it's not clear there's
actually a viable upstream anymore. Plus, we don't follow the
vendor-supplied code pattern here.

I'm doing this to reduce false positives from grep.
2018-06-10 02:30:09 +00:00
eadler
6173dc967e pmc: mark some dead functions as such
Also change a single instance of the comma operator to a semi-colon.

Reviewed by:	mmacy
2018-06-09 04:54:36 +00:00
mmacy
a7768ee751 pmc: fix logic in skipping riscv 2018-06-09 02:25:18 +00:00
mmacy
194dfe17e4 pmc: don't build on riscv where there's no kmod support 2018-06-09 02:15:45 +00:00
glebius
bb27a539de Add new functionality and syntax to cron(1) to allow to run jobs at a
given interval, which is counted in seconds since exit of the previous
invocation of the job. Example user crontab entry:

@25	sleep 10

The example will launch 'sleep 10' every 35 seconds. This is a rather
useless example above, but clearly explains the functionality.

The practical goal here is to avoid overlap of previous job invocation
to a new one, or to avoid too short interval(s) for jobs that last long
and doesn't have any point of immediate launch soon after previous run.

Another useful effect of interval jobs can be noticed when a cluster of
machines periodically communicates with a single node. Running the task
time based creates too much load on the node. Running interval based
spreads invocations across machines in cluster. Note that -j/-J won't
help in this case.

Sponsored by:	Netflix
2018-06-07 22:38:40 +00:00
mmacy
b418d2ef1e pmc: convert native to jsonl and track TSC value of samples
- add '-j' options to filter to enable converting native pmc
  log format to json lines format to enable the use of scripts
  and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
  recording nanotime when the sample is copied to a global log
  in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
  us an offset for translating the tsc values in callchain records
2018-06-07 02:03:22 +00:00
jhibbits
3a71de67c3 Add partition scheme for mpc85xx devices
Some mpc85xx devices with u-boot need MBR partitioning with a FAT boot
partition.  Since the infrastructure is already in place to have a dedicated
boot partition, this adds the necessary bits to use that infrastructure with
mpc85xx boards.

Reviewed By: nwhitehorn
Differential Revision: https://reviews.freebsd.org/D15664
2018-06-07 00:24:10 +00:00
romain
4b7f191e81 gpioctl.8: Fix man page section
Reviewed by:	manu
Approved by:	manu
2018-06-06 18:52:33 +00:00
mmacy
810fccd58a hwpmc: add summary command and further metadata extensions
metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
  to identify user vs kernel threads

fixes:
- use log cpuid to translate event id to event name

Implement rudimentary summary command to track sample
counts by thread and process name within a pmc log.

% make -j4 buildkernel >& /dev/null &
% sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15
% pmc summary foo
cpu_clk_unhalted.thread_p_any:
        idle: 138108207162
        clang-6.0: 105336158004
        sh: 72340108510
        make: 8642012963
        kernel: 7754011631
longest_lat_cache.miss:
        clang-6.0: 87502625
        sh: 40901227
        make: 5500165
        kernel: 3300099
        awk: 2000060

%  pmc summary -f ~/foo
idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003
idle: 69054
clang-6.0: 52668
sh: 36170
make: 4321
kernel: 3877
hwpmc: proc(7445): 3319
awk: 1289
xargs: 357
rand_harvestq: 181
mtree: 102
intr: 53
zfskern: 31
usb: 7
pagedaemon: 4
ntpd: 3
syslogd: 1
acpi_thermal: 1
logger: 1
syncer: 1
snmptrapd: 1
sleep: 1
idx: 17 name: longest_lat_cache.miss rate: 100003
clang-6.0: 875
sh: 409
make: 55
kernel: 33
awk: 20
hwpmc: proc(7445): 14
xargs: 9
idle: 8
intr: 3
zfskern: 2
2018-06-06 02:48:09 +00:00
mmacy
2b8bb3bc50 hwpmc: log name->pid, name->tid mappings
By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid was which when the sample was taken.

% pmc filter -T if_io_tqg -P nginx pmc.log pmc-iflib.log

% pmc filter -x -T idle pmc.log pmc-noidle.log
2018-06-05 04:26:40 +00:00
np
f53745d3d9 cxgbetool: Disallow negative values for numeric parameters.
Sponsored by:	Chelsio Communications
2018-06-04 23:17:18 +00:00
mmacy
699ee91025 pmc stat: add debug option for intermediate state 2018-06-04 22:39:22 +00:00
mmacy
666e723e89 pmc stat: fix offset of specified counter 2018-06-04 21:42:21 +00:00
jhibbits
51bc8d25a4 Set kernelname in bootconfig to the kernel file
Summary:
The kernel reads 'kernelname' to set the kern.bootfile sysctl.  By setting this,
'make installkernel' will backup the running kernel as appropriate.

Reviewed by:	nwhitehorn
Differential Revision: https://reviews.freebsd.org/D15660
2018-06-04 14:42:13 +00:00
mmacy
c0880cbd74 pmc filter: avoid spurious gcc uninitialized warning 2018-06-04 06:30:35 +00:00
mmacy
537ef3a60f pmc: add filter command
pmc filter allows the user to select event types, threads, and processes from
a sample.

% pmcstat -S unhalted_core_cycles -S llc-misses -S -S  resource_stalls.any -O pmc.log
% pmc filter -e llc-misses pmc.log pmc-llc-misses.log
% pmc filter -e unhalted_core_cycles -t 100339  pmc.log pmc-core-cycles.log
etc...
% pmcstat -R pmc-core-cycles.log -G pmc-core-cycles.stacks
2018-06-04 04:59:48 +00:00
mmacy
73041f23e1 hwpmc: support sampling both kernel and user stacks when interrupted in kernel
This adds the -U options to pmcstat which will attribute in-kernel samples
back to the user stack that invoked the system call. It is not the default,
because when looking at kernel profiles it is generally more desirable to
merge all instances of a given system call together.

Although heavily revised, this change is directly derived from D7350 by
Jonathan T. Looney.

Obtained from: jtl
Sponsored by: Juniper Networks, Limelight Networks
2018-06-04 01:10:23 +00:00
eadler
a31c3b9418 service(1): Improve manual page
* Sort options..
* Fix some typos.
* Use one Bd macro for code blocks instead of a bunch of Dl macros.
* Improve formatting.
* Clarify 'jail' argument

PR:		228552
Submitted by:	0mp
MFC After:	3 weeks
2018-06-01 04:14:16 +00:00
mmacy
4942d7a711 pmc: add list-events command 2018-06-01 00:45:59 +00:00
mmacy
bed1434e82 libpmc: allow substring for list and add function for printing event details 2018-06-01 00:45:53 +00:00
hselasky
34e8800cc7 Re-apply r190640.
- Restore local change to include <net/bpf.h> inside pcap.h.
This fixes ports build problems.
- Update local copy of dlt.h with new DLT types.
- Revert no longer needed <net/bpf.h> includes which were added
as part of r334277.

Suggested by:	antoine@, delphij@, np@
MFC after:	3 weeks
Sponsored by:	Mellanox Technologies
2018-05-31 09:11:21 +00:00
mmacy
121446bcf4 pmc stat: fix format strings for 32-bit 2018-05-30 22:03:02 +00:00
imp
ace493e034 devinfo_init() returns an errno, but doesn't set errno, so the error
message when it fails reflects some random thing rather than what it
returned. Set errno to the return value.
2018-05-30 15:08:59 +00:00
mmacy
b647eb6a42 pmc: don't break build with format issues 2018-05-30 00:06:17 +00:00
mmacy
52143988d4 pmc: silence scan-build warnings 2018-05-29 21:02:08 +00:00
mmacy
90e941d4c4 pmc: Add new sub-command structured "pmc" utility
This will manage pmc functionality with a more
manageable structure of subcommands rather than the
gradually accreted spaghetti logic of overlapping flags
that exists in pmcstat.

This is intended to ultimately have all the same functionality
as pmcannotate+pmccontrol+pmcstat. Currently it just has
"stat" and "system-stat" - counters for the process itself and counters
for the system as a whole respectively (i.e. system-stat includes kernel
threads). Note that the rusage results (page faults/context switches/
user/sys) for stat-system will not account for the system as a whole -
only for the child process specified on the command line.

Implementing stat was suggested by mjg@ and the output is based on that
from Linux's "perf stat".

% pmc stat -- make -j32 buildkernel -DNO_MODULES  -ss > /dev/null
         9598393  page faults           #       0.674 M/sec
          387085  voluntary csw         #       0.027 M/sec
          106989  involuntary csw       #       0.008 M/sec
   2763965982317  cycles
   2542953049760  instructions          #       0.920 inst/cycle
    511562750157  branches
     12917006881  branch-misses         #       2.525%
     17944429878  cache-references      #       0.007 refs/inst
      2205119560  cache-misses          #       12.289%
           43.74  real                  #       2019.72% cpu
          795.09  user                  #       1817.72% cpu
           88.35  sys                   #       202.00% cpu

% make -j32 buildkernel -DNO_MODULES  -ss > /dev/null &
% sudo pmc stat-system cat
^C             103  page faults         #       0.811 M/sec
               4  voluntary csw         #       0.031 M/sec
               0  involuntary csw       #       0.000 M/sec
   2843639070514  cycles
   2606171217438  instructions          #       0.916 inst/cycle
    522450422783  branches
     13092862839  branch-misses         #       2.506%
     18592101113  cache-references      #       0.007 refs/inst
      2562878667  cache-misses          #       13.785%
           44.85  real                  #       0.00% cpu
            0.00  user                  #       0.00% cpu
            0.00  sys                   #       0.00% cpu
2018-05-29 20:28:34 +00:00
araujo
7183e8f4e5 Simplify macros EFPRINTF and EFFLUSH. [0]
Also stdarg(3) says that each invocation of va_start() must be paired
with a corresponding invocation of va_end() in the same function. [1]

Reported by:	Coverity
CID:		1194318[0] and 1194332[1]
Discussed with:	jhb
MFC after:	4 weeks.
Sponsored by:	iXsystems Inc.
Differential Revision:	https://reviews.freebsd.org/D15548
2018-05-29 01:46:00 +00:00
dteske
61459d6c71 sysrc(8): Test variable names for invalid characters
PR:		bin/187461
Reported by:	ebay@looksharp.net
MFC after:	4 weeks
X-MFC-to:	stable/11 (after 11.2-R)
Sponsored by:	Smule, Inc.
2018-05-28 23:34:23 +00:00
hselasky
512f24f465 Buildfix for GCC after r334277.
MFC after:	1 month
Sponsored by:	Mellanox Technologies
2018-05-28 09:41:44 +00:00
hselasky
30f165e2d7 MFV r333789: libpcap 1.9.0 (pre-release)
MFC after:	1 month
Sponsored by:	Mellanox Technologies
2018-05-28 08:12:18 +00:00
vangyzen
28ec75c409 bhyve: guarantee NUL termination
Use strlcpy to guarantee NUL termination of the path to a
virtio console socket.

Reported by:	Coverity
CID:		1362874
Sponsored by:	Dell EMC
2018-05-28 03:09:09 +00:00
vangyzen
975f33299b bhyve: fix small memory leak in virtio console
Reported by:	Coverity
CID:		1363284
Sponsored by:	Dell EMC
2018-05-28 03:05:01 +00:00
vangyzen
abe545ff03 tcpdrop: ensure NUL termination of a string
strncpy did not guarantee NUL termination of the "stack" string.
Use strlcpy instead.  While I'm here, avoid unnecessary memset
and strnlen calls.

Reported by:	Coverity
CID:		1381035
Sponsored by:	Dell EMC
2018-05-28 01:58:49 +00:00
mmacy
3fe632a10d pmcstat: suppress uninitialized warning of event 2018-05-27 17:49:27 +00:00
mmacy
0224710b2f pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.
2018-05-26 19:29:19 +00:00
mmacy
274069bbf6 Revert r334242 "pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the"
because of squash commit messages
2018-05-26 19:26:19 +00:00
mmacy
2e56e89840 pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.

Squashed commit of the following:

commit 4459d43eff815bec08ccc5533dbe5de846f03128
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Sat May 26 00:06:31 2018 -0700

    libpmc: fix pmu function signatures for non amd64

commit a2cb8bbc586c65d41f9b291430a2261ec67b59fe
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:38:11 2018 -0700

    pmcstat: fix indentation of usage

commit f686954b15ff56a833ac80404898977cb80a265b
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:49 2018 -0700

    pmclog(3): add callchain and pmcallocatedyn, remove pcsample

commit 73e13a0d2e9498c81c150d14d022050cee7511bb
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:00 2018 -0700

    pmclog.h: GC pcsample field

commit 3e93ffd65da641fa657539dad3c48e281f8b5798
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:05:57 2018 -0700

    hwpmc: make Intel core CPUs use external event tables

commit 634f5fae1e1644ac324003136c66cd9c619d1c93
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:00:06 2018 -0700

    pmclog: update log record types, bump PMC_MAJOR
    - explicitly make log record types a multiple of 8 bytes
    - hook in pmu event types for pmc_allocate records
    - remove references to no longer PCSAMPLE record

commit 83d84fcd2d65bdf6ddcb2e155a22f0cfa2a9c225
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 21:52:10 2018 -0700

    libpmc: add support for having vendor table driven pmc_allocate

commit 9e6ad63c40c2fce8404847ace5078ca6cb33a736
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:11:33 2018 -0700

    hwpmc_core: add accessors for EVSEL & UMASK, make IAP_UMASK useful to user

commit 859dceb93daa6419a48c794db99b6758e5b041c9
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:09:45 2018 -0700

    pmcstat: update usage and man page as well as make -L consistent with pmccontrol

commit 79c7d8597e28c2eb13f5f9113e65ec2792ca57b1
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 18:07:03 2018 -0700

    pmu_util: add support for all current intel event keywords

commit d8089c7f6a6c8527f38324252b1ffb47004694c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:45:00 2018 -0700

    add description for new arguments

commit 058336740bab53c62ec88a3a026ea848cf3878c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:38:15 2018 -0700

    libpmc: move pmu_events table and pmu_utils out of libpmcstat so that they can be used by pmc_allocate

commit 049b66b382e2f833c3f47bc8df9e750cb265709f
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:12:41 2018 -0700

    pmcstat: hook pmu_events counter description utility routines in

commit f5e01e7b37a691dc045e1aa16b3ebdd162515de8
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:11:59 2018 -0700

    pmu_events: add utility routines for listing counters and their descriptions

commit cba4d4f8907f772279f86f18f915e0d74d33ac56
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:09:50 2018 -0700

    pmu-events: expand out skylake regex to simplify string matches
2018-05-26 18:12:50 +00:00
araujo
2d1010154c We don't need check if str is NULL as free(3) will handle NULL
argument.

Reported by:	kib@
2018-05-25 19:12:30 +00:00