Commit Graph

208 Commits

Author SHA1 Message Date
Emmanuel Vadot
93c4369096 pkgbase: Put more binaries/lib in runtime
Move some needed binaries/libs from FreeBSD-utilities to FreeBSD_runtime.
This is everything needed to boot to multiuser with FreeBSD-rc installed.

MFC after:	2 weeks
Sponsored by:	Beckhoff Automation GmbH & Co. KG
Differential Revision:	https://reviews.freebsd.org/D33435
2021-12-21 10:17:27 +01:00
Kyle Evans
4c14980baa grep: fix/remove references to -P
-P in gnugrepland means PCRE, which we do not support.  We may eventually
support it if onigmo ends up getting imported as a more performant regex
implementation, and we can re-add it properly in these places (and more)
when that time comes.

The optstr change is a functional nop; the case was not explicitly handled,
thus ending in usage() anyways.

Reported by:	Vladimir Misev (via twitter)
2021-11-10 00:42:42 -06:00
Mariusz Zaborski
24c681a7f6 grep: fix combination of quite and count flag
When the quite (-q) flag is provided, we don't expect any output.
Currently, the behavior is broken:
$ grep -cq flag util.c
1

$ grep -cs flag util.c
55

First of all, we print a number to stdout. Secondly, it just returns
0 or 1 (which is unexpected). GNU grep with c and q flags doesn't
print anything.

Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D31108
2021-07-09 14:09:14 +02:00
Mateusz Piotrowski
16e0391f8e Fix zgrep --version
"zgrep --version" is expected to print the version information in the
same way as "zgrep -V". However, the case handling the --version flag
is never reached, so "zgrep --version" prints:
    zgrep: missing pattern
instead of:
    grep (BSD grep, GNU compatible) 2.6.0-FreeBSD

Reviewed by:	yuripv
Approved by:	yuripv (src)
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D29813
2021-04-18 10:20:11 +02:00
Mateusz Piotrowski
be6b8b7a3a grep: Fix an incorrect description of the -C flag
It seems that the number of lines is no longer an optional parameter to
the -C flag. Document it accordingly both in the manual page and the
usage message.

Reviewed by:	yuripv
Approved by:	yuripv
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D28509
2021-04-17 22:35:15 +02:00
Alex Richardson
81c3f64110 usr.bin/grep: Fix Address OOB read error
I found this when compiling all the bootstrap tools with -fsanitize=addres:

==65590==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62d000008400 at pc 0x000000473053 bp 0x7ffc1c7dd910 sp 0x7ffc1c7dd0b8
READ of size 32769 at 0x62d000008400 thread T0
    #0 0x473052 in regexec (/local/scratch/alr48/cheri/build/freebsd-amd64-build/local/scratch/alr48/cheri/freebsd/amd64.amd64/tmp/legacy/bin/grep+0x473052)
    #1 0x4c9cf3 in procline /local/scratch/alr48/cheri/freebsd/usr.bin/grep/util.c:539:8
    #2 0x4c8687 in procfile /local/scratch/alr48/cheri/freebsd/usr.bin/grep/util.c:379:18
    #3 0x4c6596 in main /local/scratch/alr48/cheri/freebsd/usr.bin/grep/grep.c:714:8

0x62d000008400 is located 0 bytes to the right of 32768-byte region [0x62d000000400,0x62d000008400)
allocated by thread T0 here:
    #0 0x493d5d in malloc (/local/scratch/alr48/cheri/build/freebsd-amd64-build/local/scratch/alr48/cheri/freebsd/amd64.amd64/tmp/legacy/bin/grep+0x493d5d)
    #1 0x4cad75 in grep_malloc /local/scratch/alr48/cheri/freebsd/usr.bin/grep/util.c:656:13
    #2 0x4c8129 in procfile /local/scratch/alr48/cheri/freebsd/usr.bin/grep/util.c
    #3 0x4c6596 in main /local/scratch/alr48/cheri/freebsd/usr.bin/grep/grep.c:714:8

SUMMARY: AddressSanitizer: heap-buffer-overflow (/local/scratch/alr48/cheri/build/freebsd-amd64-build/local/scratch/alr48/cheri/freebsd/amd64.amd64/tmp/legacy/bin/grep+0x473052) in regexec

Reviewed By:	kevans
MFC after:	1 week
2021-02-09 17:13:32 +00:00
Kyle Evans
3e2d96ac97 grep: fix -A handling in conjunction with -m match limitation
The basic issue here is that grep, when given -m 1, would stop all
line processing once it hit the match count and exit immediately.  The
problem with exiting immediately is that -A processing only happens when
subsequent lines are processed and do not match.

The fix here is relatively easy; when bsdgrep matches a line, it resets
the 'tail' of the matching context to the value supplied to -A and
dumps anything that's been queued up for -B. After the current line has
been printed and tail is reset, we check our mcount and do what's
needed. Therefore, at the time that we decide we're doing nothing, we
know that 'tail' of the context is correct and we can simply continue
on if there's still more to pick up.

With this change, we still bail out immediately if there's been no -A
flag. If -A was supplied, we signal that we should continue on. However,
subsequent lines will not even bothere to try and process the line.  We
have reached the match count, so even if the next line would match then
we must process it if it hadn't. Thus, the loop in procfile() can
short-circuit and just process the line as a non-match until
procmatches() indicates that it's safe to stop.

A test has been added to reflect both that we should be picking up the
next line and that the next line should be considered a non-match even
if it should have been.

PR:		253350
MFC-after:	3 days
2021-02-08 12:41:22 -06:00
Kyle Evans
2373acbbb7 grep: turn off -w if -x is specified
-x overcomes -w in gnugrep, and it should here as well.  Flip it off as
needed to avoid confusing other parts of grep.
2021-02-04 20:59:43 -06:00
Kyle Evans
f823c6dc73 grep: fix null pattern and empty pattern file behavior
The null pattern semantics were terrible because I tried to match gnugrep,
but I got it wrong.  Let's unwind that:

- The null pattern should match every line if neither -w nor -x.
- The null pattern should match empty lines if -x.
- The null pattern should not match any lines if -w.

The first two will stop processing (shortcut) even if additional patterns
are specified. In any other case, we will continue processing other
patterns.  If no other patterns are specified beside a null pattern, then
we match if neither -w nor -x or set and do not match if either of those
are specified.

The justification for -w is that it should match on a whole word, but the
null pattern deos not have a whole word to match on.

Empty pattern files should never match anything, and more importantly, -v
should cause everything to be written.

PR:		253209
MFC-after:	4 days
2021-02-04 20:59:42 -06:00
Martin Tournoij
f850fd2496 grep: fix LINKS in Makefile
zstdegrep was listed twice, instead of zstdfgrep

Pull Request:	https://github.com/freebsd/freebsd-src/pull/450
2021-01-18 11:22:48 +01:00
Kyle Evans
d1c965f143 grep: tests: stop testing for a nonexistent version of grep
Differential Revision:	https://reviews.freebsd.org/D27732
2020-12-25 15:16:33 -06:00
Kyle Evans
8aff76fb37 build: remove the option to build gnugrep
Unconditionally install bsdgrep as grep, bootstrap or not. Remove all
build glue and stop installing both gnugrep and libgnuregex now that
all consumers of the latter are gone.

Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D27732
2020-12-25 15:14:17 -06:00
Kyle Evans
df546c3b73 grep: replace the internal queue with a ring buffer
We know up front how many items we can have in the queue (-B/Bflag), so
pay the cost of those particular allocations early on.

The reduced queue maintenance overhead seemed to yield about an ~8%
improvement for my earlier `grep -C8 -r closefrom .` test.

MFC after:	2 weeks
2020-12-09 05:27:45 +00:00
Kyle Evans
281412ce7b grep: tests: stop expecting a failure of gnuext w/ bsdgrep
libregex now supports these and we no longer offer to not link against
libregex.
2020-12-09 05:12:04 +00:00
Bryan Drewery
2dfa4b66b3 fts_read: Handle error from a NULL return better.
This is addressing cases such as fts_read(3) encountering an [EIO]
from fchdir(2) when FTS_NOCHDIR is not set.  That would otherwise be
seen as a successful traversal in some of these cases while silently
discarding expected work.

As noted in r264201, fts_read() does not set errno to 0 on a successful
EOF so it needs to be set before calling it.  Otherwise we might see
a random error from one of the iterations.

gzip is ignoring most errors and could be improved separately.

Reviewed by:	vangyzen
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D27184
2020-12-08 23:38:26 +00:00
Kyle Evans
6e5a5dc03d bsdgrep: don't link against libregex for bootstrap
r368355 removed the GNU_GREP_COMPAT knob (off by default) and forgot that
bsdgrep may be built/used for bootstrap on some systems.

All base uses should strive to use only POSIX-compliant expressions anyways
and we haven't had libregex by default here up to this point, so just don't
do that if we're bootstrapping.

Note that the resulting binary has the wrong `grep -V` information as it
falsely claims to be GNU compatible, but it is only for bootstrap.

Reported by:	GitHub cross-builds via yuripv
2020-12-06 17:45:42 +00:00
Kyle Evans
7c2f310f6d Retire GNU_GREP_COMPAT knob
This was introduced and then disabled by default primarily to avoid dealing
with bugs in libgnuregex. rS363823 switched to using libregex for it, so
let's just rip the option out now so we can make sure we're getting tested
with libregex via bsdgrep.

Reviewed by:	emaste
Differential Revision:	https://reviews.freebsd.org/D27476
2020-12-05 02:21:58 +00:00
Eugene Grosbein
1032f7b82b bzgrep: make flag --no-filename work
PR:		248813
MFC after:	1 week
2020-11-20 09:56:46 +00:00
Fernando Apesteguía
5be3f744b4 grep(1): Add more EXAMPLES
* Add more EXAMPLES covering flags: -A, -B, -c, -f, -i, -H, -l, -q, -R, -w
* While here, change existing wording to use the imperative (remove "To
  find")
* Reword first example to be consistent with how grep(1) understand
  words (-w)

Approved by:	manpages (bcr@)
Differential Revision:		https://reviews.freebsd.org/D27264
2020-11-19 18:58:15 +00:00
Eric van Gyzen
63c8336d4d zgrep: fix exit status with multiple files
zgrep should exit with success when given multiple files and the
pattern is found in at least one file.  Prior to this change,
it would exit with success only if the pattern was found in _every_ file.

Reviewed by:	dab ngie
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D26616
2020-10-01 21:48:22 +00:00
Alex Richardson
86ce536521 Fix -Wpointer-sign warnings in usr.bin/grep
Reviewed By:	kevans
Differential Revision: https://reviews.freebsd.org/D26479
2020-09-21 09:03:37 +00:00
Alex Richardson
4f971ddf33 Always install usr.bin/grep as grep when bootstrapping
We have to bootstrap grep when cross-building from macOS/Linux.
2020-08-07 16:04:01 +00:00
Gordon Bergling
88d241831d grep(1): correct typos for 'if a name patches' to 'if a name matches'
PR:		237635
Submitted by:	durin42 <raf atdurin42 dot com>
Reviewed by:	kevans
Approved by:	kevans
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D25994
2020-08-07 16:01:05 +00:00
Kyle Evans
cab7d341dc bsdgrep: switch to libregex for GNU_GREP_COMPAT
libregex is incomplete, but it's a bit less buggy than the in-base
libgnuregex and mostly OK.

While here, rename -DIWTH_GNU -> -DWITH_GNU_COMPAT; the option implies
that we're compatible with the GNU counterpart, not that we're including GNU
anything.
2020-08-04 02:47:24 +00:00
Craig Leres
c4cbf1fbab Fix some regressions with the zgrep(1) wrapper.
- Handle whitespace with long flags that take arguments:

	echo 'foo bar' > test
	zgrep --regexp='foo bar' test

 - Do not hang reading from stdin with patterns in a file:

	echo foobar > test
	echo foo > pattern
	zgrep -f pattern test
	zgrep --file=pattern test

 - Handle any flags after -e:

	echo foobar > test
	zgrep -e foo --ignore-case < test

These two are still outstanding problems:

 - Does not handle flags that take an argument if there is no
   whitespace:

	zgrep -enfs /etc/rpc

 - When more than one -e pattern used matching should occur for all
   patterns (similar to multiple patterns supplied with -f file).
   Instead only the last pattern is used for matching:

	zgrep -e rex -e nfs /etc/rpc

   (This problem is masked in the unpatched version by the "any
   flags after -e" problem.)

Add tests for the above problems.

Update the mange and add references to gzip(1) and zstd(1) and also
document the remaining known problems.

PR:		247126
Approved by:	markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D25613
2020-07-20 23:57:53 +00:00
Simon J. Gerraty
2c9a9dfc18 Update Makefile.depend files
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22494
2019-12-11 17:37:53 +00:00
Simon J. Gerraty
5ab1c5846f Add Makefile.depend.options
Leaf directories that have dependencies impacted
by options need a Makefile.depend.options file
to avoid churn in Makefile.depend

DIRDEPS for cases such as OPENSSL, TCP_WRAPPERS etc
can be set in local.dirdeps-options.mk
which can add to those set in Makefile.depend.options

See share/mk/dirdeps-options.mk

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22469
2019-12-11 17:37:37 +00:00
Kyle Evans
38325e2ab8 bsdgrep(1): various fixes of empty pattern/exit code/-c behavior
When an empty pattern is encountered in the pattern list, I had previously
broken bsdgrep to count that as a "match all" and ignore any other patterns
in the list. This commit rectifies that mistake, among others:

- The -v flag semantics were not quite right; lines matched should have been
  counted differently based on whether the -v flag was set or not. procline
  now definitively returns whether it's matched or not, and interpreting
  that result has been kicked up a level.
- Empty patterns with the -x flag was broken similarly to empty patterns
  with the -w flag. The former is a whole-line match and should be more
  strict, only matching blank lines. No -x and no -w will will match the
  empty string at the beginning of each line.
- The exit code with -L was broken, w.r.t. modern grep. Modern grap will
  exit(0) if any file that didn't match was output, so our interpretation
  was simply backwards. The new interpretation makes sense to me.

Tests updated and added to try and catch some of this.

This misbehavior was found by autoconf while fixing ports found in PR 229925
expecting either a more sane or a more GNU-like sed.

MFC after:	1 week
2019-09-25 17:14:43 +00:00
Kyle Evans
aef64c62c2 bsdgrep(1): add some basic tests for some GNU Extension support
These will be expanded later as I come up with good test cases; for now,
these seem to be enough to trigger bugs in base gnugrep and expose missing
features in bsdgrep.
2019-09-03 18:32:29 +00:00
Kyle Evans
25385eb325 grep: Move lone 'r'grep case into the adjacent switch
This 'r' case should have belonged to the switch in the first place, but
I had somehow missed the switch when initially adding the rgrep link. The
zgrep script later came along and faithfully left this case standing alone,
so we will now go ahead and join it.

Nearby comment also adjusted a tad bit for wording and style.

Reported by:	Daniel Ebdrup
MFC after:	3 days
2019-06-02 02:38:44 +00:00
Kyle Evans
26e3f9681b Fix bsdgrep manpage clobbering grep(1) with default build options
The default build should install bsdgrep(1) and grep(1), with the latter
being gnugrep(1). WITH_BSD_GREP flips this situation such that we have
gnugrep(1) and grep(1), with the latter being bsdgrep(1).

Changes to start installing the zgrep script out of usr.bin/grep
inadvertently altered the default build such that grep(1) was being
installed, and it was bsdgrep(1). Correct the typo.

Reported by:	bcran
MFC after:	3 days
2019-01-09 02:47:07 +00:00
Mateusz Piotrowski
6ddfef55bd Adjust formatting of grep and zgrep manual pages.
grep(1) changes:
 - Pet mandoc & igor.
 - Stylize the text more with macros when appropriate.
 - Stylize equal signs in long options (e.g., "--color=auto") with
   the "Cm" macro as suggested by mdoc(7).
 - Add missing arguments to --exlude, --exclude-dir, --include and
   --include-dir.
 - Remove a duplicate entry for the --context flag.
 - Use a list in the EXAMPLES sections to make it easier to tell
   which paragraphs belong to which example.
 - Cross reference zgrep(1).

zgrep(1) changes:
 - Fix Nd.
 - Split synopsis into paragraphs for readability.
 - Cross reference bzip(1), grep(1) and xz(1).

Reviewed by:	bcr
Approved by:	mat (mentor)
Differential Revision:	https://reviews.freebsd.org/D16779
2018-08-20 22:23:59 +00:00
Kyle Evans
031f92f512 bsdgrep(1): Remove redundant initialization; unconditionally assigned later 2018-06-15 03:31:30 +00:00
Kyle Evans
be13c0f967 bsdgrep(1): Some more int -> bool conversions and name changes
Again motivated by upcoming work to rewrite a bunch of this- single-letter
variable names and slightly misleading variable names ("lastmatches" to
indicate that the last matched) are not helpful.
2018-06-09 18:11:46 +00:00
Kyle Evans
d1a920b48f bsdgrep(1): Evict character sequence that moved in
Reported by:	trasz
2018-06-08 12:58:55 +00:00
Kyle Evans
bd60b9b499 bsdgrep(1): Slooowly peel away the chunky onion
(or peel off the band-aid, whatever floats your boat)

This addresses two separate issues:

1.) Nothing within bsdgrep actually knew whether it cared about line numbers
  or not.

2.) The file layer knew nothing about the context in which it was being
  called.

#1 is only important when we're *not* processing line-by-line. #2 is
debatably a good idea; the parsing context is only handy because that's
where we store current offset information and, as of this commit, whether or
not it needs to be line-aware.
2018-06-08 01:25:07 +00:00
Kyle Evans
66f780aee0 bsdgrep(1): Don't initialize fts_flags twice
Admittedly, this is a clang-scan complaint... but it wasn't wrong. fts_flags
is initialized by all cases in the switch(), which should be fairly obvious.
Annotate this anyways.
2018-06-07 18:38:48 +00:00
Kyle Evans
40f0e0b100 bsdgrep(1): whoops, garbage collect the now write-only variable 2018-06-07 18:36:12 +00:00
Kyle Evans
cbfff13fa2 bsdgrep(1): Do some less dirty things with return types
Neither procfile nor grep_tree return anything meaningful to their callers.
None of the callers actually care about how many lines were matched in all
of the files they processed; it's all about "did anything match?"

This is generally just a light refactoring to remind me of what actually
matters as I'm rewriting these bits to care less about 'stuff'.
2018-06-07 18:27:58 +00:00
Baptiste Daroussin
30dc95029e Remove NLS support from BSD grep
GNU grep as in actually in base does not have any translations support
compiled in, so no functionnality loss.

We do support 193 locales in base, we will never catch up on that number of
translation with bsd grep.

Removing NLS support make bsd grep consistent with the other binaries in base
which are not translated, and also reduce a little bit the code.

Reviewed by:	kevans
Approved by:	kevans
Discussed with:	kevans @BSDCan
Differential Revision:	https://reviews.freebsd.org/D15682
2018-06-06 23:12:35 +00:00
Bryan Drewery
920a817c78 Fix exit code for mismatches after r333013.
The -c flag still does the wrong thing versus the older version due to
lack of pipefail support.

Reported by:	antoine
2018-05-24 22:15:47 +00:00
Kyle Evans
f0fb94abca Standardize SPDX tag on files I've added 2018-05-09 16:52:28 +00:00
Kyle Evans
4b3c64f722 Remove "All Rights Reserved" on files that I hold sole copyright on
See r333391 for more detail; in summary: it holds no weight and may be
removed.
2018-05-09 16:44:19 +00:00
Kyle Evans
24a656c291 bsdgrep: Allow "-" to be passed to -f to mean "standard input"
A version of this patch was originally sent to me by se@, matching behavior
from newer versions of GNU grep.

While there have been some differences of opinion on whether stdin should be
closed or not after depleting it in process of -f, I've opted to leave stdin
open and just let the later matching stuff fail and result in a no-match.
I'm not married to the current behavior- it was generally chosen since we
are adopting this in particular from GNU grep, and I would like to stay
consistent without a strong argument to the contrary. The current behavior
isn't technically wrong, it's just fairly unfriendly to the developer-user
of grep that may not realize their usage is trivially invalid.

Submitted by:	se
2018-05-08 03:53:46 +00:00
Kyle Evans
a2584d1b34 bsdgrep: annihilate our in-tree TRE, previously disabled by default
It was an old TRE that had plenty of bugs and no performance gain over
regex(3). I disabled it by default in r323615, and there was some confusion
about what the knob does- likely due to poor naming on my part- to the tune
of "well, it sounds like it should speed things up" (mentioned by multiple
people).

To compound this, I have no intention of maintaining a second regex
implementation. If someone would like to step up and volunteer to maintain a
lean-and-mean implementation for grep, this is OK, but we have very few
volunteers to maintain even our primary regex implementation.
2018-05-04 03:13:25 +00:00
Kyle Evans
42fd147f79 zgrep(1): Note that -r/-R are not currently supported.
This is better behavior than just silently doing the wrong thing. We do not
currently have plans to support -r/-R with the compression-enabled greps.

Reported by:	jilles
2018-05-03 02:56:13 +00:00
Kyle Evans
f2f0b02bdc bsdgrep: Adjust a missed NLS reference that was invalidated by recent work
Submitted by:	Dan McGregor <dan.mcgregor@usask.ca>
2018-05-02 15:45:31 +00:00
Baptiste Daroussin
6a88262d6a zgrep.sh: Add forgotten ';' and remove set -e which prevents looping over files
not matching
2018-04-25 21:01:02 +00:00
Baptiste Daroussin
1e5b97628b zgrep.sh: remove now useless shift 2018-04-25 20:55:18 +00:00
Baptiste Daroussin
9fdb4cfa89 zgrep: small improvements
* Use slightly more efficient method to determine the name of the program
called [1]
* Use nicer form to loop over arguments [1]
* add special support for --version along with -V previously added by kevans

Reported by:	jilles@ [1]
2018-04-25 20:52:17 +00:00