We know up front how many items we can have in the queue (-B/Bflag), so
pay the cost of those particular allocations early on.
The reduced queue maintenance overhead seemed to yield about an ~8%
improvement for my earlier `grep -C8 -r closefrom .` test.
MFC after: 2 weeks
This is addressing cases such as fts_read(3) encountering an [EIO]
from fchdir(2) when FTS_NOCHDIR is not set. That would otherwise be
seen as a successful traversal in some of these cases while silently
discarding expected work.
As noted in r264201, fts_read() does not set errno to 0 on a successful
EOF so it needs to be set before calling it. Otherwise we might see
a random error from one of the iterations.
gzip is ignoring most errors and could be improved separately.
Reviewed by: vangyzen
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D27184
r368355 removed the GNU_GREP_COMPAT knob (off by default) and forgot that
bsdgrep may be built/used for bootstrap on some systems.
All base uses should strive to use only POSIX-compliant expressions anyways
and we haven't had libregex by default here up to this point, so just don't
do that if we're bootstrapping.
Note that the resulting binary has the wrong `grep -V` information as it
falsely claims to be GNU compatible, but it is only for bootstrap.
Reported by: GitHub cross-builds via yuripv
This was introduced and then disabled by default primarily to avoid dealing
with bugs in libgnuregex. rS363823 switched to using libregex for it, so
let's just rip the option out now so we can make sure we're getting tested
with libregex via bsdgrep.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D27476
* Add more EXAMPLES covering flags: -A, -B, -c, -f, -i, -H, -l, -q, -R, -w
* While here, change existing wording to use the imperative (remove "To
find")
* Reword first example to be consistent with how grep(1) understand
words (-w)
Approved by: manpages (bcr@)
Differential Revision: https://reviews.freebsd.org/D27264
zgrep should exit with success when given multiple files and the
pattern is found in at least one file. Prior to this change,
it would exit with success only if the pattern was found in _every_ file.
Reviewed by: dab ngie
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D26616
libregex is incomplete, but it's a bit less buggy than the in-base
libgnuregex and mostly OK.
While here, rename -DIWTH_GNU -> -DWITH_GNU_COMPAT; the option implies
that we're compatible with the GNU counterpart, not that we're including GNU
anything.
- Handle whitespace with long flags that take arguments:
echo 'foo bar' > test
zgrep --regexp='foo bar' test
- Do not hang reading from stdin with patterns in a file:
echo foobar > test
echo foo > pattern
zgrep -f pattern test
zgrep --file=pattern test
- Handle any flags after -e:
echo foobar > test
zgrep -e foo --ignore-case < test
These two are still outstanding problems:
- Does not handle flags that take an argument if there is no
whitespace:
zgrep -enfs /etc/rpc
- When more than one -e pattern used matching should occur for all
patterns (similar to multiple patterns supplied with -f file).
Instead only the last pattern is used for matching:
zgrep -e rex -e nfs /etc/rpc
(This problem is masked in the unpatched version by the "any
flags after -e" problem.)
Add tests for the above problems.
Update the mange and add references to gzip(1) and zstd(1) and also
document the remaining known problems.
PR: 247126
Approved by: markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D25613
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22494
Leaf directories that have dependencies impacted
by options need a Makefile.depend.options file
to avoid churn in Makefile.depend
DIRDEPS for cases such as OPENSSL, TCP_WRAPPERS etc
can be set in local.dirdeps-options.mk
which can add to those set in Makefile.depend.options
See share/mk/dirdeps-options.mk
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22469
When an empty pattern is encountered in the pattern list, I had previously
broken bsdgrep to count that as a "match all" and ignore any other patterns
in the list. This commit rectifies that mistake, among others:
- The -v flag semantics were not quite right; lines matched should have been
counted differently based on whether the -v flag was set or not. procline
now definitively returns whether it's matched or not, and interpreting
that result has been kicked up a level.
- Empty patterns with the -x flag was broken similarly to empty patterns
with the -w flag. The former is a whole-line match and should be more
strict, only matching blank lines. No -x and no -w will will match the
empty string at the beginning of each line.
- The exit code with -L was broken, w.r.t. modern grep. Modern grap will
exit(0) if any file that didn't match was output, so our interpretation
was simply backwards. The new interpretation makes sense to me.
Tests updated and added to try and catch some of this.
This misbehavior was found by autoconf while fixing ports found in PR 229925
expecting either a more sane or a more GNU-like sed.
MFC after: 1 week
These will be expanded later as I come up with good test cases; for now,
these seem to be enough to trigger bugs in base gnugrep and expose missing
features in bsdgrep.
This 'r' case should have belonged to the switch in the first place, but
I had somehow missed the switch when initially adding the rgrep link. The
zgrep script later came along and faithfully left this case standing alone,
so we will now go ahead and join it.
Nearby comment also adjusted a tad bit for wording and style.
Reported by: Daniel Ebdrup
MFC after: 3 days
The default build should install bsdgrep(1) and grep(1), with the latter
being gnugrep(1). WITH_BSD_GREP flips this situation such that we have
gnugrep(1) and grep(1), with the latter being bsdgrep(1).
Changes to start installing the zgrep script out of usr.bin/grep
inadvertently altered the default build such that grep(1) was being
installed, and it was bsdgrep(1). Correct the typo.
Reported by: bcran
MFC after: 3 days
grep(1) changes:
- Pet mandoc & igor.
- Stylize the text more with macros when appropriate.
- Stylize equal signs in long options (e.g., "--color=auto") with
the "Cm" macro as suggested by mdoc(7).
- Add missing arguments to --exlude, --exclude-dir, --include and
--include-dir.
- Remove a duplicate entry for the --context flag.
- Use a list in the EXAMPLES sections to make it easier to tell
which paragraphs belong to which example.
- Cross reference zgrep(1).
zgrep(1) changes:
- Fix Nd.
- Split synopsis into paragraphs for readability.
- Cross reference bzip(1), grep(1) and xz(1).
Reviewed by: bcr
Approved by: mat (mentor)
Differential Revision: https://reviews.freebsd.org/D16779
Again motivated by upcoming work to rewrite a bunch of this- single-letter
variable names and slightly misleading variable names ("lastmatches" to
indicate that the last matched) are not helpful.
(or peel off the band-aid, whatever floats your boat)
This addresses two separate issues:
1.) Nothing within bsdgrep actually knew whether it cared about line numbers
or not.
2.) The file layer knew nothing about the context in which it was being
called.
#1 is only important when we're *not* processing line-by-line. #2 is
debatably a good idea; the parsing context is only handy because that's
where we store current offset information and, as of this commit, whether or
not it needs to be line-aware.
Admittedly, this is a clang-scan complaint... but it wasn't wrong. fts_flags
is initialized by all cases in the switch(), which should be fairly obvious.
Annotate this anyways.
Neither procfile nor grep_tree return anything meaningful to their callers.
None of the callers actually care about how many lines were matched in all
of the files they processed; it's all about "did anything match?"
This is generally just a light refactoring to remind me of what actually
matters as I'm rewriting these bits to care less about 'stuff'.
GNU grep as in actually in base does not have any translations support
compiled in, so no functionnality loss.
We do support 193 locales in base, we will never catch up on that number of
translation with bsd grep.
Removing NLS support make bsd grep consistent with the other binaries in base
which are not translated, and also reduce a little bit the code.
Reviewed by: kevans
Approved by: kevans
Discussed with: kevans @BSDCan
Differential Revision: https://reviews.freebsd.org/D15682
A version of this patch was originally sent to me by se@, matching behavior
from newer versions of GNU grep.
While there have been some differences of opinion on whether stdin should be
closed or not after depleting it in process of -f, I've opted to leave stdin
open and just let the later matching stuff fail and result in a no-match.
I'm not married to the current behavior- it was generally chosen since we
are adopting this in particular from GNU grep, and I would like to stay
consistent without a strong argument to the contrary. The current behavior
isn't technically wrong, it's just fairly unfriendly to the developer-user
of grep that may not realize their usage is trivially invalid.
Submitted by: se
It was an old TRE that had plenty of bugs and no performance gain over
regex(3). I disabled it by default in r323615, and there was some confusion
about what the knob does- likely due to poor naming on my part- to the tune
of "well, it sounds like it should speed things up" (mentioned by multiple
people).
To compound this, I have no intention of maintaining a second regex
implementation. If someone would like to step up and volunteer to maintain a
lean-and-mean implementation for grep, this is OK, but we have very few
volunteers to maintain even our primary regex implementation.
This is better behavior than just silently doing the wrong thing. We do not
currently have plans to support -r/-R with the compression-enabled greps.
Reported by: jilles
* Use slightly more efficient method to determine the name of the program
called [1]
* Use nicer form to loop over arguments [1]
* add special support for --version along with -V previously added by kevans
Reported by: jilles@ [1]
- The --exclude{,-dir} and --include{,-dir} directives now match GNU
behavior of being processed in order and latest matching directive wins
- --label was previously not really documented, and -L and -l did not
indicate that --label applied to them
- The flags listed as being extensions to POSIX spec were not updated with
the removal of compression-related flags
MFC after: 1 week
Compression was removed so #2 goes away and everything else needs renumbered
to match, and the usage string was also updated due to removed options.
X-MFC-With: r332995
Compression support is now handled by an external script, remove it from the
bsdgrep(1) utility.
This removes the support for -Z -J -X and -M
Note: that it matches the changes in newer GNU grep
Reviewed by: kevans
Approved by: kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D15197
Import the wrapper script from zstdgrep (written by wiz@netbsd.org)
Modify it to support more than just zstd (adding support for gzip,
lzma, xz and bzip2)
Write a simple manpage dedicated for it.
Only use that new wrapper both for gnu grep and bsd grep
Next step will be removing code related to compression format from bsdgrep
Reviewed by: kevans
Approved by: kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D15193
Prior to r332851:
* --exclude always win out over --include
* --exclude-dir always wins out over --include-dir
r332851 broke that behavior, resulting in:
* First of --exclude, --include wins
* First of --exclude-dir, --include-dir wins
As it turns out, both behaviors are wrong by modern grep standards- the
latest rule wins. e.g.:
`grep --exclude foo --include foo 'thing' foo`
foo is included
`grep --include foo --exclude foo 'thing' foo`
foo is excluded
As tested with GNU grep 3.1.
This commit makes bsdgrep follow this behavior.
Reported by: se
There's no point checking for a bunch of file modes if we're not a
practicing believer of DIR_SKIP or DEV_SKIP.
This also reduces some style violations that were particularly ugly looking
when browsing through.
Split the matching and non-matching cases out into their own functions to
reduce future complexity. As the name implies, procmatches will eventually
process more than one match itself in the future.
