13770 Commits

Author SHA1 Message Date
bapt
2f45771b4b Use a script wrapper for <compress>grep
Import the wrapper script from zstdgrep (written by wiz@netbsd.org)

Modify it to support more than just zstd (adding support for gzip,
lzma, xz and bzip2)

Write a simple manpage dedicated for it.

Only use that new wrapper both for gnu grep and bsd grep

Next step will be removing code related to compression format from bsdgrep

Reviewed by:	kevans
Approved by:	kevans
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D15193
2018-04-25 13:23:58 +00:00
emaste
deddde63b9 kdump: simplify/remove per-arch #ifdefs
It is acceptable for syscallabi to map SV_ABI to SYSDECODE_ABI on all
architectures; libsysdecode will return not-found sentinel values if
it does not have a syscall name or errno mapping for a given
architecture.

Also, use __LP64__ for the SV_ILP32 -> SYSDECODE_ABI_LINUX32 mapping,
for any future 32- on 64-bit linuxulator implementation.

Reviewed by:	jhb
Sponsored by:	Turing Robotic Industries Inc.
2018-04-24 19:23:26 +00:00
kevans
5cbcde6d07 bsdgrep: Fix build failure WITHOUT_LZMA (incorrect bracket placement)
Submitted by:	sbruno
Reported by:	sbruno
2018-04-22 23:51:24 +00:00
bcran
d8988e8715 Update account and given names in committers-src.dot and calendar.freebsd
I have changed my given name from Bruce to Rebecca, and my FreeBSD account
from brucec to bcran.
Update committers-src.dot and calendar.freebsd to show these changes.

Reviewed by:	rrs
Differential Revision: https://reviews.freebsd.org/D15125
2018-04-22 20:47:16 +00:00
kevans
334fa0aab4 bsdgrep: Use grep_strdup instead of grep_malloc+strcpy 2018-04-21 14:58:45 +00:00
kevans
985dd72ad9 bsdgrep: Fix --include/--exclude ordering issues
Prior to r332851:
* --exclude always win out over --include
* --exclude-dir always wins out over --include-dir

r332851 broke that behavior, resulting in:
* First of --exclude, --include wins
* First of --exclude-dir, --include-dir wins

As it turns out, both behaviors are wrong by modern grep standards- the
latest rule wins. e.g.:

`grep --exclude foo --include foo 'thing' foo`
foo is included

`grep --include foo --exclude foo 'thing' foo`
foo is excluded

As tested with GNU grep 3.1.

This commit makes bsdgrep follow this behavior.

Reported by:	se
2018-04-21 13:46:07 +00:00
kevans
564bfbdaa2 bsdgrep: if chain => switch
This makes some of this a little easier to follow (in my opinion).
2018-04-21 01:42:02 +00:00
kevans
ad41916557 bsdgrep: More trivial cleanup/style cleanup
We can avoid branching for these easily reduced patterns
2018-04-21 01:33:13 +00:00
kevans
00c090fd14 bsdgrep: Some light cleanup
There's no point checking for a bunch of file modes if we're not a
practicing believer of DIR_SKIP or DEV_SKIP.

This also reduces some style violations that were particularly ugly looking
when browsing through.
2018-04-21 01:02:35 +00:00
kevans
0dde510be5 bsdgrep: Break procmatches down a little bit more
Split the matching and non-matching cases out into their own functions to
reduce future complexity. As the name implies, procmatches will eventually
process more than one match itself in the future.
2018-04-20 18:06:03 +00:00
kevans
49168f0878 bsdgrep: Add some TODOs for future work on operating on chunks 2018-04-20 03:29:06 +00:00
kevans
68c7a8f091 bsdgrep: Clean up procmatches a little bit 2018-04-20 03:11:51 +00:00
kevans
1c4340834f bsdgrep: Split match processing out of procfile
procfile is getting kind of hairy, and it's not going to get better as we
correct some more bits that assume we process one line at a time.
2018-04-20 03:08:46 +00:00
emaste
8d5207ff27 chpass: reject change/expiry dates beyond y2106
The pwd.db and spwd.db files store the change and expire dates as
unsigned 32-bit ints, which overflow in 2106.  Reject larger values for
now, until the introduction of a v5 password database.

i386 has 32-bit time_t and so dates beyond y2038 are already rejected by
mktime.

PR:		227589
Reviewed by:	lidl
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-04-19 12:50:49 +00:00
cem
cec6f35b11 quota(1): Fix calculation overflow and underflow
For very large quotas, do the multiplication as a 64 bit value to avoid
overflow.

For very small block sizes (smaller than DEV_BSIZE), multiple first
before dividing by block size to avoid underflow.

PR:		227496
Submitted by:	Per Andersson <pa AT chalmers.se>
Sponsored by:	Dell EMC Isilon
2018-04-16 19:33:04 +00:00
ed
d178bf0959 Add RFC 5424 syslog message output to syslogd.
- Move all of the code responsible for transmitting log messages into a
  separate function, fprintlog_write().
- Instead of manually modifying a list of iovecs, add a structure
  iovlist with some helper functions.
- Alter the F_FORW (UDP message forwarding) case to also use iovecs like
  the other cases. Use sendmsg() instead of sendto().
- In the case of F_FORW, truncate the message to a size dependent on the
  address family (AF_INET, AF_INET6), as proposed by RFC 5426.
- Move all traditional message formatting into fprintlog_bsd(). Get rid
  of some of the string copying and snprintf()'ing. Simply emit more
  iovecs to get the job done.
- Increase ttymsg()'s limit of 7 iovecs to 32. Add a definition for this
  limit, so it can be reused by iovlist.
- Add fprintlog_rfc5424() to emit RFC 5424 formatted log entries.
- Add a "-O" command line option to enable RFC 5424 formatting. It would
  have been nicer if we supported "-o rfc5424", just like on NetBSD.
  Unfortunately, the "-o" flag is already used for a different purpose
  on FreeBSD.
- Don't truncate hostnames in the RFC 5424 case, as suggested by that
  specific RFC.

For people interested in using this, this feature can be enabled by
adding the following line to /etc/rc.conf:

  syslogd_flags="-s -O rfc5424"

Differential Revision:	https://reviews.freebsd.org/D15011
2018-04-15 08:34:16 +00:00
kevans
2e980f1b2c dtc(1): Update to upstream 006664a
Highlights:

- Passing "-" to -o will now cause output to go to stdout
- Path-based syntactic sugar for overlays is now accepted. This looks like:

/dts-v1/;
/plugin/;

&{/soc} {
    sid: eeprom@1c14000 {
        compatible = "allwinner,sun8i-h3-sid";
        reg = <0x1c14000 0x400>;
        status = "okay";
    };
};

MFC after:	3 days
2018-04-13 18:04:51 +00:00
bapt
1377c1931e reinstall the roff(7) manpage
When we had both groff and mandoc in base, we decided to keep the roff(7)
manpage from groff. when remoing groff, we forgot to install the mandoc version
instead.

This fixes it.

Reported by:	trasz
MFC after:	1 week
2018-04-13 09:04:31 +00:00
benno
10246e837a Fix a conditional that got mucked up.
Sponsored by:	iXsystems, Inc.
2018-04-12 17:16:13 +00:00
benno
4d1ff9c758 Remove a debugging printf that crept in.
Sponsored by:	iXsystems, Inc.
2018-04-12 17:00:36 +00:00
benno
1ebccb19a5 Check the return value of fseek.
Reported by:	Coverity
CID:		1388267
Sponsored by:	iXsystems, Inc.
2018-04-12 15:50:19 +00:00
benno
19e7651c74 Add the ability to specify absolute and relative offsets to size partitions.
To create hybrid boot media we want to specify a partition at a known location.
This extends the syntax of size partitions to include an optional offset that
can be absolute or relative. It also introduces validation to make sure that
this hasn't resulted in overlapping partitions. I haven't added this to the
file and process partition specifications yet but the mechanics are designed
such that if someone comes up with a good way of specifying the offset it
will be fairly easy to add in.

Reviewed by:	imp
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14916
2018-04-12 15:47:47 +00:00
dim
4b91ca5ee1 Regenerate clang man page after upstream change to document the possible
values for the -std= option.

Noticed by:	Steve Kargl
Obtained from:	https://reviews.llvm.org/rL329827
MFC after:	3 days
2018-04-11 18:39:47 +00:00
kevans
b75e4698e9 head(1): Provide long options
Provide long options --bytes and --lines to match -c and -n respectively.
This improves head(1)'s compatibility with its GNU counterpart in a sensible
way.

Reviewed by:	eadler (previous version)
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D14139
2018-04-10 14:44:07 +00:00
kevans
6df8601ce6 tail(1): Address mandoc concern (space before punctuation after macro)
X-MFC-With:	r332372
2018-04-10 14:42:24 +00:00
kevans
58813c2c55 tail(1): Add some long options
Add --blocks, --bytes, and --lines long options for -b, -c, and -n
respectively. This improves tail(1)'s compatibility with its GNU counterpart
in a straightforward way.

Reviewed by:	eadler (earlier version)
MFC after:	3 days
2018-04-10 14:27:27 +00:00
tuexen
72609e5c8e Don't show the number of currently established SCTP associations,
since this is not monotonically increasing. It's number can be
derived from the other counters shown.

MFC after:	3 days
2018-04-10 08:19:14 +00:00
benno
2f0aaeb453 Add the etdump utility for dumping El Torito boot catalog information.
This can be used to check existing images but will be used in the future to
find EFI ESP images placed in El Torito catalogs so they can be used for
hybrid boot purposes.

Reviewed by:	imp (code), sbruno (man page), bcr (man page)
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D14952
2018-04-03 21:08:10 +00:00
emaste
8d951aca16 vtfontcvt: allow .bdf characters less than full height
Sponsored by:	The FreeBSD Foundation
2018-04-03 18:43:00 +00:00
ericbsd
65e4b69acf Adding Eric Turgeon (ericbsd) to calendar.freebsd
Approved by: bapt (mentor)
2018-03-30 13:37:33 +00:00
jeff
5e244328ad Implement several enhancements to NUMA policies.
Add a new "interleave" allocation policy which stripes pages across
domains with a stride or width keeping contiguity within a multi-page
region.

Move the kernel to the dedicated numbered cpuset #2 making it possible
to assign kernel threads and memory policy separately from user.  This
also eliminates the need for the complicated interrupt binding code.

Add a sysctl API for viewing and manipulating domainsets.  Refactor some
of the cpuset_t manipulation code using the generic bitset type so that
it can be used for both.  This probably belongs in a dedicated subr file.

Attempt to improve the include situation.

Reviewed by:	kib
Discussed with:	jhb (cpuset parts)
Tested by:	pho (before review feedback)
Sponsored by:	Netflix, Dell/EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D14839
2018-03-29 02:54:50 +00:00
cem
9d420f4029 timeout.1: Try to improve clarity
Minor rewordings, markup fixes or enhancements, and some typo fixes.  Add a few
sentences clarifying the special zero duration.

PR:		227012
Submitted by:	Mateusz Piotrowski (0mp@) (earlier version)
2018-03-29 02:13:58 +00:00
jhb
481b9cd955 Reformat the enum of syscall argument types.
List enum values on separate lines to minimize diffs as new types are
added.  Split the enum values up into groups and use some simple sorting
within groups (scalar enums are sorted by size, then base, all other
groups are generally sorted alphabetically).

No functional change.
2018-03-29 00:12:50 +00:00
jhb
0624ccfe44 Rename ClouduABIFDSFlags to CloudABIFDSFlags. 2018-03-29 00:04:07 +00:00
jeff
3e6c614462 Document new NUMA related syscalls and utility options.
Sponsored by:	Netflix, Dell/EMC Isilon
2018-03-24 23:58:44 +00:00
jtl
80fab35507 Bump netstat.1's .Dd after r331347. 2018-03-22 09:43:15 +00:00
jtl
a93bdf6963 Add the "TCP Blackbox Recorder" which we discussed at the developer
summits at BSDCan and BSDCam in 2017.

The TCP Blackbox Recorder allows you to capture events on a TCP connection
in a ring buffer. It stores metadata with the event. It optionally stores
the TCP header associated with an event (if the event is associated with a
packet) and also optionally stores information on the sockets.

It supports setting a log ID on a TCP connection and using this to correlate
multiple connections that share a common log ID.

You can log connections in different modes. If you are doing a coordinated
test with a particular connection, you may tell the system to put it in
mode 4 (continuous dump). Or, if you just want to monitor for errors, you
can put it in mode 1 (ring buffer) and dump all the ring buffers associated
with the connection ID when we receive an error signal for that connection
ID. You can set a default mode that will be applied to a particular ratio
of incoming connections. You can also manually set a mode using a socket
option.

This commit includes only basic probes. rrs@ has added quite an abundance
of probes in his TCP development work. He plans to commit those soon.

There are user-space programs which we plan to commit as ports. These read
the data from the log device and output pcapng files, and then let you
analyze the data (and metadata) in the pcapng files.

Reviewed by:	gnn (previous version)
Obtained from:	Netflix, Inc.
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D11085
2018-03-22 09:40:08 +00:00
cem
82710b55b6 Implement getrandom(2) and getentropy(3)
The general idea here is to provide userspace programs with well-defined
sources of entropy, in a fashion that doesn't require opening a new file
descriptor (ulimits) or accessing paths (/dev/urandom may be restricted
by chroot or capsicum).

getrandom(2) is the more general API, and comes from the Linux world.
Since our urandom and random devices are identical, the GRND_RANDOM flag
is ignored.

getentropy(3) is added as a compatibility shim for the OpenBSD API.

truss(1) support is included.

Tests for both system calls are provided.  Coverage is believed to be at
least as comprehensive as LTP getrandom(2) test coverage.  Additionally,
instructions for running the LTP tests directly against FreeBSD are provided
in the "Test Plan" section of the Differential revision linked below.  (They
pass, of course.)

PR:		194204
Reported by:	David CARLIER <david.carlier AT hardenedbsd.org>
Discussed with:	cperciva, delphij, jhb, markj
Relnotes:	maybe
Differential Revision:	https://reviews.freebsd.org/D14500
2018-03-21 01:15:45 +00:00
mjg
58bd3bad5d Update my calendar entry to be more specific now that UTF8 turns out to be fine 2018-03-19 00:25:10 +00:00
dteske
48e117e8c9 Use full month in dpv(3), figpar(3), and bsdconfig(8) manuals
Reported by:	maxim
2018-03-14 19:09:06 +00:00
fernape
06bcb6ba44 Add myself (fernape) to calendar.freebsd
As indicated in Committers guide Chapter 6, point 9
"Optional: Update Ports with Personal Information"

Approved by:	tcberner
Differential Revision:	https://reviews.freebsd.org/D14653
2018-03-11 17:21:48 +00:00
trasz
abf7c30dfd Make iscsictl(1) display "Disabled" status for disabled sessions.
MFC after:	2 weeks
2018-03-10 14:33:00 +00:00
asomers
ccec95ca4c tftpd: Flush files as soon as they are fully received
On an RRQ, tftpd doesn't exit as soon as it's finished receiving a file.
Instead, it waits five seconds just in case the client didn't receive the
server's last ACK and decides to resend the final DATA packet.
Unfortunately, this created a 5 second delay from when the client thinks
it's done sending the file, and when the file is available for other
processes.

Fix this bug by closing the file as soon as receipt is finished.

PR:			157700
Reported by:		Barry Mishler <barry_mishler@yahoo.com>
MFC after:		3 weeks
2018-03-09 23:25:18 +00:00
eadler
ba41b4e2ac calendars: update Judaic calendar to 2018+
This was generated by

∴hebcal --years 10 -r 2018 | awk -F '[/\t]' '{print $3 "/" $1 "/" $2
"*\t" $4}'

MFC After:	1 week
2018-03-08 05:28:43 +00:00
brooks
ca6208b512 Fix build post r330299 2018-03-02 23:31:55 +00:00
brooks
da9d8a0895 Don't declare union semun in userspace unless _WANT_SEMUN is defined.
POSIX explicitly states that the application must declare union semun.
This makes no sense, but it is what it is.  This brings us into line
with Linux, MacOS/Darwin, and NetBSD.

In a ports exp-run a moderate number of ports fail due to a lack of
approprate autotools-like discovery mechanisms or local patches.  A
commit to address them will follow shortly.

PR:		224300, 224443 (exp-run)
Reviewed by:	emaste, jhb, kib
Exp-run by:	antoine
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14492
2018-03-02 22:32:53 +00:00
sjg
490c4b4d3b Update to bmake-201802222
Fixes segfault in Var_Set if val is NULL
Don't treat .info as warning with -W
2018-03-02 01:53:50 +00:00
cem
4956af63b1 seq(1): Consistently include 'last' for non-integers
The source of error is a rounded increment being too large and thus the loop
steps slightly past 'last'.  Perform a final comparison using the formatted
string values (truncated precision) to determine if we still need to print
the 'last' value.

PR:		217149
Submitted by:	Fernando Apesteguía <fernando.apesteguia AT gmail.com>,
		Yuri Pankov <yuripv AT icloud.com> (earlier version)
Reported by:	Martijn Dekker <mcdutchie AT hotmail.com>
Sponsored by:	Dell EMC Isilon
2018-02-27 22:01:40 +00:00
trasz
bd43ba9e2b .Xr rctl(8) and cpuset(1).
PR:		225935
Submitted by:	D. Ebdrup <debdrup at gmail.com> (earlier version)
MFC after:	2 weeks
2018-02-26 18:04:17 +00:00
emaste
b3fd6ddf73 ministat: disallow negative variance / nan Stddev
With all values identical it was possible for Var() to return a negative
value due to limited floating point precision, resulting in "nan"
reported as Stddev.

Variance cannot actually be negative, so just return 0.  We can later
investigate alternate algorithms for calculating variance to reduce the
effect of catastrophic cancellation here.

Reported by:	Arshan Khanifar <arshankhanifar_gmail.com>
Approved by:	phk
Sponsored by:	The FreeBSD Foundation
2018-02-21 15:54:23 +00:00