Commit Graph

1102 Commits

Author SHA1 Message Date
Mark Johnston
e505460228 Import the netdump client code.
This is a component of a system which lets the kernel dump core to
a remote host after a panic, rather than to a local storage device.
The server component is available in the ports tree. netdump is
particularly useful on diskless systems.

The netdump(4) man page contains some details describing the protocol.
Support for configuring netdump will be added to dumpon(8) in a future
commit. To use netdump, the kernel must have been compiled with the
NETDUMP option.

The initial revision of netdump was written by Darrell Anderson and
was integrated into Sandvine's OS, from which this version was derived.

Reviewed by:	bdrewery, cem (earlier versions), julian, sbruno
MFC after:	1 month
X-MFC note:	use a spare field in struct ifnet
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D15253
2018-05-06 00:38:29 +00:00
John Baldwin
8ce99bb405 Properly do a deep copy of the ioctls capability array for fget_cap().
fget_cap() tries to do a cheaper snapshot of a file descriptor without
holding the file descriptor lock.  This snapshot does not do a deep
copy of the ioctls capability array, but instead uses a different
return value to inform the caller to retry the copy with the lock
held.  However, filecaps_copy() was returning 1 to indicate that a
retry was required, and fget_cap() was checking for 0 (actually
'!filecaps_copy()').  As a result, fget_cap() did not do a deep copy
of the ioctls array and just reused the original pointer.  This cause
multiple file descriptor entries to think they owned the same pointer
and eventually resulted in duplicate frees.

The only code path that I'm aware of that triggers this is to create a
listen socket that has a restricted list of ioctls and then call
accept() which calls fget_cap() with a valid filecaps structure from
getsock_cap().

To fix, change the return value of filecaps_copy() to return true if
it succeeds in copying the caps and false if it fails because the lock
is required.  I find this more intuitive than fixing the caller in
this case.  While here, change the return type from 'int' to 'bool'.

Finally, make filecaps_copy() more robust in the failure case by not
copying any of the source filecaps structure over.  This avoids the
possibility of leaking a pointer into a structure if a similar future
caller doesn't properly handle the return value from filecaps_copy()
at the expense of one more branch.

I also added a test case that panics before this change and now passes.

Reviewed by:	kib
Discussed with:	mjg (not a fan of the extra branch)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D15047
2018-04-17 18:07:40 +00:00
Kristof Provost
98c5f9884e pf tests: Basic ioctl validation tests
Validate the DIOCRADDTABLES and DIOCRDELTABLES ioctls with invalid size
values. All of these requests should fail.

MFC after:	1 week
2018-04-06 15:03:48 +00:00
Ruslan Bukin
cde6fa2857 Add new shared library -- libopencsd.
OpenCSD is an ARM CoreSight(tm) trace packets decoder.

- Connect libopencsd to the arm64 build.
- Install opencsd headers to /usr/include/opencsd/

Sponsored by:	DARPA, AFRL
2018-04-04 14:31:56 +00:00
Jonathan T. Looney
2529f56ed3 Add the "TCP Blackbox Recorder" which we discussed at the developer
summits at BSDCan and BSDCam in 2017.

The TCP Blackbox Recorder allows you to capture events on a TCP connection
in a ring buffer. It stores metadata with the event. It optionally stores
the TCP header associated with an event (if the event is associated with a
packet) and also optionally stores information on the sockets.

It supports setting a log ID on a TCP connection and using this to correlate
multiple connections that share a common log ID.

You can log connections in different modes. If you are doing a coordinated
test with a particular connection, you may tell the system to put it in
mode 4 (continuous dump). Or, if you just want to monitor for errors, you
can put it in mode 1 (ring buffer) and dump all the ring buffers associated
with the connection ID when we receive an error signal for that connection
ID. You can set a default mode that will be applied to a particular ratio
of incoming connections. You can also manually set a mode using a socket
option.

This commit includes only basic probes. rrs@ has added quite an abundance
of probes in his TCP development work. He plans to commit those soon.

There are user-space programs which we plan to commit as ports. These read
the data from the log device and output pcapng files, and then let you
analyze the data (and metadata) in the pcapng files.

Reviewed by:	gnn (previous version)
Obtained from:	Netflix, Inc.
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D11085
2018-03-22 09:40:08 +00:00
Ruslan Bukin
30b3274fff Add new shared library -- libipt.
libipt is the Intel Processor Trace (Intel PT) packets decoder.

- Include libipt to amd64 build.
- Install libipt headers to /usr/include/libipt/

Sponsored by:	DARPA, AFRL
2018-03-21 14:37:04 +00:00
Kyle Evans
d5a390e6c7 Move /boot/overlays to /boot/dtb/overlays
The former is fairly vague; these are FDT overlays to be applied to the
running system, so /boot/dtb is a sensible location to put it without
cluttering up /boot/dtb even further if desired.
2018-03-19 16:16:12 +00:00
Alan Somers
c60fdff77d Commit missing file from r330696
MFC after:	3 weeks
X-MFC-With:	330696
2018-03-09 23:17:29 +00:00
Hans Petter Selasky
e808190a59 Add kernel and userspace code to dump the firmware state of supported
ConnectX-4/5 devices in mlx5core.

The dump is obtained by reading a predefined register map from the
non-destructive crspace, accessible by the vendor-specific PCIe
capability (VSC). The dump is stored in preallocated kernel memory and
managed by the mlx5tool(8), which communicates with the driver using a
character device node.

The utility allows to store the dump in format
    <address> <value>
into a file, to reset the dump content, and to manually initiate the
dump.

A call to mlx5_fwdump() should be added at the places where a dump
must be fetched automatically. The most likely place is right before a
firmware reset request.

Submitted by:	kib@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-08 15:21:56 +00:00
Devin Teske
5bf5ca772c Introduce dwatch(1) as a tool for making DTrace more useful
Reviewed by:	markj, gnn, bdrewery (earlier version)
Relnotes:	yes
Sponsored by:	Smule, Inc.
Differential Revision:	https://reviews.freebsd.org/D10006
2018-03-06 23:44:19 +00:00
Kyle Evans
3eae2a2e40 Add 'usr.bin/seq' to tests mtree after r330086 2018-02-27 22:22:23 +00:00
Alan Somers
4b40bdbd1f Add tests for lagg(4) and other cloned network interfaces
Unfortunately, most of the tests are disabled because they fairly frequently
trigger panics.

MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
2018-02-23 18:18:42 +00:00
Alan Somers
2fae26bd8b Add the ZFS test suite
It was originally written by Sun as part of the STF (Solaris test framework).
They open sourced it in OpenSolaris, then HighCloud partially ported it to
FreeBSD, and Spectra Logic finished the port.  We also added many testcases,
fixed many broken ones, and converted them all to the ATF framework.  We've had
help along the way from avg, araujo, smh, and brd.

By default most of the tests are disabled.  Set the disks Kyua variable to
enable them.

Submitted by:	asomers, will, justing, ken, brd, avg, araujo, smh
Sponsored by:	Spectra Logic Corp, HighCloud
2018-02-23 16:31:00 +00:00
Alan Somers
ea9c2614cf Add mtree entry for 329275
MFC after:	3 weeks
X-MFC-With:	329275
Sponsored by:	Spectra Logic Corp
2018-02-14 21:02:38 +00:00
Warner Losh
b11df8a4a3 Add /boot/lua. 2018-02-13 17:42:10 +00:00
Kyle Evans
25f0135c49 Add /boot/overlays to runtime pkg, fix distrib-dirs METALOG generation
/boot/overlays was recently added without belonging to a package. It's only
used by bootloaders at the moment, so add it to the 'runtime' package to get
added with ubldr and friends.

Fix distrib-dirs METALOG generation while we're here. History elsewhere
seems to indicate that bapt@ fixed this to pull in all attributes from
mtrees while generating the METALOG. This fix got clobbered somewhere later,
so restore it.

Reviewed by:	bapt, gjb
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D13996
2018-01-23 17:59:06 +00:00
Kyle Evans
b37f6c9805 Add libregex, connect it to the build
libregex is a regex(3) implementation intended to feature GNU extensions and
any other non-POSIX compliant extensions that are deemed worthy.

These extensions are separated out into a separate library for the sake of
not cluttering up libc further with them as well as not deteriorating the
speed (or lack thereof) of the libc implementation.

libregex is implemented as a build of the libc implementation with LIBREGEX
defined to distinguish this from a libc build. The reasons for
implementation like this are two-fold:

1.) Maintenance- This reduces the overhead induced by adding yet another
regex implementation to base.

2.) Ease of use- Flipping on GNU extensions will be as simple as linking
against libregex, and POSIX-compliant compilations can be guaranteed with a
REG_POSIX cflag that should be ignored by libc/regex and disables extensions
in libregex. It is also easier to keep REG_POSIX sane and POSIX pure when
implemented in this fashion.

Tests are added for future functionality, but left disconnected for the time
being while other testing is done.

Reviewed by:	cem (previous version)
Differential Revision:	https://reviews.freebsd.org/D12934
2018-01-22 02:44:41 +00:00
Kyle Evans
db180ae55c stand: Add /boot/overlays to allow separation of overlays from base FDT
This matches directory structure used commonly in Linux-land, and it's
cleaner than mixing overlays into the existing module paths. Overlays are
still mixed in by specifying fdt_overlays in loader.conf(5).

Reviewed by:	manu
Differential Revision:	https://reviews.freebsd.org/D13922
2018-01-18 04:58:54 +00:00
Dimitry Andric
c79126f2e4 Merge ^/head r327624 through r327885. 2018-01-12 18:23:35 +00:00
Kyle Evans
12cea332f1 vmstat(8): Hook up NetBSD tests
The NetBSD tests for vmstat are basically just a smoke test, ensuring that
executing `vmstat` and `vmstat -s` exit successfully. This is more than we
test now, so go with it.
2018-01-11 16:04:04 +00:00
Kyle Evans
de45c289b9 awk(1): Add necessary bits for connecting tests, but leave disconnected
The NetBSD test suite has 24 tests for awk, and we pass exactly 4 of them.
Add the necessary pieces for interested parties to easily connect the
tests and run them, but leave them disconnected for the time being.

Some of these tests outright segfault in our awk, others just exhibit the
wrong behavior.
2018-01-11 05:36:13 +00:00
Dimitry Andric
4b49587c3d Merge ^/head r327341 through r327623. 2018-01-06 16:13:17 +00:00
Jilles Tjoelker
9d75d6c9d5 find: Link tests to the build 2017-12-31 19:24:13 +00:00
Eitan Adler
5539cb324e mtree: remove /etc/skel
We use /usr/share/skel instead of /etc/skel. The existence of /etc/skel
has confused people.

PR:		46062 (submitted 2002-12-07)
PR:		218897
Submitted by:	carl@slackerbsd.org
Submitted by:	asv@inhio.net
2017-12-31 07:25:55 +00:00
Dimitry Andric
27228b49fb Update clang versioned dir in mtree files. 2017-12-20 20:28:40 +00:00
Dimitry Andric
14767bd616 Follow-up to r325967, which removed /etc/casper, by also removing it
from BSD.root.dist, so it does not get created again on installworld.
2017-12-12 22:21:20 +00:00
Eitan Adler
fbc88a6f35 sponge(1): revert
I did a complete buildworld and test... with the program disconnected
from the tree. Revert the change for now.

(this keeps the change to .arclint which is still correct)

Wearing:	my pointhat
2017-12-06 02:47:46 +00:00
Eitan Adler
8d4a7aab40 sponge(1): fix my tests
Reviewed by:	kevans
2017-12-05 04:43:39 +00:00
Alan Somers
95639a80ef dc(1): fix input of non-decimal fractional numbers
Inputting fractional non-decimal numbers has never worked correctly in our
OpenBSD-derived dc(1). It truncates the input to a number of decimal places
equal to the number of hexadecimal (or whatever base) places given on the
input. That's unacceptable, because many numbers require more precision to
represent in base 10 than in their original bases.

Fix this bug by using as many decimal places as needed to represent the
input, up to the maximum of the global scale factor.

This has one mildly surprising side effect: the scale of a number entered in
non-decimal mode will no longer necessarily equal the number of hexadecimal
(or whatever base) places given on the input. I think that's an acceptable
behavior change, given that inputting fractional non-decimal numbers never
worked in the first place, and the man page doesn't specify whether trailing
zeros on the input should affect a number's scale.

PR:		206230
Reported by:	nibbana@gmx.us
Reviewed by:	pfg
Differential Revision:	https://reviews.freebsd.org/D13336
2017-12-05 04:22:35 +00:00
Mark Johnston
04006780d9 Complete support for dtrace's -x setenv option.
This allows one to override the environment for processes created with
dtrace -c. By default, the environment is inherited.

This support was originally merged from illumos in r249367 but was lost
when the commit was later reverted and then brought back piecemeal.

Reported by:	Samuel Lepetit <slepetit@apple.com>
MFC after:	2 weeks
2017-12-03 16:57:28 +00:00
Kristof Provost
4fbebc7472 Add IPSec tests in tunnel mode
Some IPSec in tunnel mode allowing to test multiple IPSec
configurations.  These tests are reusing the jail/vnet scripts from pf
tests for generating complex network.

Submitted by:	olivier@
Differential Revision:	https://reviews.freebsd.org/D13017
2017-12-03 13:52:35 +00:00
Dimitry Andric
d4419f6fa8 Upgrade our copies of clang, llvm, lldb and libc++ to r319231 from the
upstream release_50 branch.  This corresponds to 5.0.1 rc2.

MFC after:	2 weeks
2017-12-03 12:14:34 +00:00
Alan Somers
013953eb5f Add basic tests for ctfconvert(1), fold(1) and rs(1)
Add basic command line parsing test coverage for these utilities.  The tests
were automatically generated based on their man pages.  These tests can be
expanded by hand for more thorough coverage.  The aim is to generate very
basic amount of test coverage for all the utilities in the base system.

Tests generated via: https://github.com/shivansh/smoketestsuite/

Submitted by:	shivansh
Reviewed by:	asomers
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D12424
2017-11-27 20:01:58 +00:00
Alan Somers
396c556d77 Add ATF tests for head(1)
Submitted by:	Fred Schlecter <https://github.com/fjs-github>
Reviewed by:	asomers, jilles
MFC after:	3 weeks
Differential Revision:	https://github.com/freebsd/freebsd/pull/127
2017-11-20 22:55:02 +00:00
Konstantin Belousov
9898800172 Remove xlint(1).
xlint is currently a fossil.  We have much more useful and alive tools
to do now what xlint did twenty years ago.

I did not cleared some stuff which makes lint operational, in
sys/x86/include and sys/sys, but I might do it as followup.  The
x86/include/ucontext.h and _types.h hacks made to please lint was the
main reason for my initial proposal to classify xlint as obsolete and
to remove it.

Also I do not intend to clear sccs ids.

Reviewed by:	bapt, brooks, emaste, jhb, pfg
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D13015
2017-11-16 14:37:18 +00:00
Bryan Drewery
939d033cab Disconnect libpathconv tests since they require external perl and do not work with kyua.
This reverts r325192 and is due to libpathconv being connected in r325186.

Reported by:	ngie
Sponsored by:	Dell EMC Isilon
2017-10-31 19:52:30 +00:00
Bryan Drewery
ae160963d8 Fix installworld/distrib-dirs for pathconv after r325186.
Sponsored by:	Dell EMC Isilon
2017-10-31 01:43:36 +00:00
Kristof Provost
1d6f5f214a pf: Basic automated test using VIMAGE
If VIMAGE is present we can start jails with their own pf instance. This
makes it fairly easy to run tests.
For example, this basic test verifies that drop/pass and icmp
classification works. It's a basic sanity test for pf, and hopefully an
example on how to write more pf tests.

The tests are skipped if VIMAGE is not enabled.

This work is inspired by the GSoC work of Panagiotes Mousikides.

Differential Revision:	https://reviews.freebsd.org/D12580
2017-10-06 20:43:14 +00:00
Jeremie Le Hen
e415aa2846 Remove rcmds.
If they are still needed, you can find them in the net/bsdrcmds port.

This was proposed June, 20th and approved by various committers [1].
They have been marked as deprecated on CURRENT in r320644 [2] on July, 4th.
Both stable/11 and release/11.1 contain the deprecation notice (thanks to
allanjude@).

Note that ruptime(1)/rwho(1)/rwhod(8) were initially thought to be part of
rcmds but this was a mistake and those are therefore NOT removed.

[1] https://lists.freebsd.org/pipermail/freebsd-arch/2017-June/018239.html
[2] https://svnweb.freebsd.org/base?view=revision&revision=320644

Reviewed by:	bapt, brooks
Differential Revision:	https://reviews.freebsd.org/D12573
2017-10-06 08:43:14 +00:00
Baptiste Daroussin
e6340c5d05 Remove unneeded locales and alias man directories
In base, locales (and encoding) specific directories are not used
by any tool. Just remove them.

While here also remove the cat page directory for openssl
2017-09-26 05:43:55 +00:00
Baptiste Daroussin
05572d356b Remove the cat pages directory now that catman(1) is gone 2017-09-25 21:23:49 +00:00
Alan Somers
014404db1a Add basic tests for chflags, mkdir, rcp, and rmdir
Add basic command line parsing test coverage for these utilities.  The tests
were automatically generated based on their man pages.  These tests can be
expanded by hand for more thorough coverage.  The aim is to generate very
basic amount of test coverage for all the utilities in the base system.

Submitted by:	shivansh
Reviewed by:	asomers, brooks
MFC after:	3 weeks
Sponsored by:	Google, Inc (GSoC 2017)
Differential Revision:	https://reviews.freebsd.org/D12036
2017-09-07 16:54:47 +00:00
Jilles Tjoelker
2cc32af06f sh: Add tests for sh -c that already pass.
PR:		220587
Submitted by:	Ryan Moeller
2017-08-12 19:17:48 +00:00
Enji Cooper
fcb60eb0bb Unconditionally install etc/mtree/BSD.debug.dist again
r279248 unconditionally installed BSD.debug.dist for ease-of-developer-use.
Restore the previous behavior.

While here, add a comment to note that this is intentional to avoid accidental
future removal.

MFC after:	2 months
MFC with:	r321444
2017-07-25 00:28:23 +00:00
Dimitry Andric
2fef18f836 Merge ^/head r320994 through r321238. 2017-07-19 19:43:10 +00:00
Kristof Provost
4d7709ddf6 pfctl parser tests
Copy the most important test cases from OpenBSD's corresponding
src/regress/sbin/pfctl, those that run pfctl on a test input file and check
correctness of its output. We have also added some new tests using the same
format.

The tests consist of a collection of input files (pf*.in) and
corresponding output files (pf*.ok). We run pfctl -nv on the input
files and check that the output matches the output files. If any
discrepancy is discovered during future development in the source
tree, we know that a regression bug has been introduced into the tree.

Submitted by:	paggas
Sponsored by:	Google, Inc (GSoC 2017)
Differential Revision:	https://reviews.freebsd.org/D11322
2017-07-15 19:22:01 +00:00
Dimitry Andric
d2043ca373 Merge ^/head r320573 through r320970. 2017-07-13 22:01:38 +00:00
Kyle Evans
b5ddde39ac Add some basic tests for hexdump(1)'s various output flags. Formatting
tests are omitted for this initial run as there are still some bugs to work
out there.

This covers -s flag testing on devices and non-devices that would have
caught breakage found in PR 219173 as well as other subtle breakage caused
locally.

Reviewed by:	cem, ngie
Approved by:	cem (acting co-mentor)
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D11279
2017-07-13 03:52:54 +00:00
Warner Losh
a94a63f0a6 An MMC/SD/SDIO stack using CAM
Implement the MMC/SD/SDIO protocol within a CAM framework. CAM's
flexible queueing will make it easier to write non-storage drivers
than the legacy stack. SDIO drivers from both the kernel and as
userland daemons are possible, though much of that functionality will
come later.

Some of the CAM integration isn't complete (there are sleeps in the
device probe state machine, for example), but those minor issues can
be improved in-tree more easily than out of tree and shouldn't gate
progress on other fronts. Appologies to reviews if specific items
have been overlooked.

Submitted by: Ilya Bakulin
Reviewed by: emaste, imp, mav, adrian, ian
Differential Review: https://reviews.freebsd.org/D4761

merge with first commit, various compile hacks.
2017-07-09 16:57:24 +00:00
Enji Cooper
f37852c173 Add tests to help verify Links functionality for .../contrib/tzdata/backwards
MFC after:	1 month
MFC with:	r320702
2017-07-06 04:30:06 +00:00