Commit Graph

35 Commits

Author SHA1 Message Date
kevans
17df4eb706 bsdgrep: add a primitive literal matcher
fgrep/grep -F will error out at runtime if compiled with a regex(3)
that does not define REG_NOSPEC or REG_LITERAL. glibc is one such regex(3)
implementation, and as it turns out they don't support literal matching at
all.

Provide a primitive literal matcher for use with glibc and other
implementations that don't support literal matching so that we don't
completely lose fgrep/grep -F if compiled against libgnuregex on stable/10,
stable/11, or other systems that we don't necessarily support.

This is a wholly unoptimized implementation with no plans to optimize it as
of now. This is due to both its use-case being primarily on unsupported
systems in the near-distant future and that it's reinventing the wheel that
we already have available as a feature of regex(3).

Reviewed by:	cem, emaste, ngie
Approved by:	emaste (mentor)
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D12056
2017-08-24 01:23:33 +00:00
emaste
6b50dcbb8c bsdgrep: correct assumptions to prepare for chunking
Correct a couple of minor BSD grep assumptions that are valid for line
processing but not future chunk-based processing.

Submitted by:	Kyle Evans <kevans91@ksu.edu>
Reviewed by:	bapt, cem
Differential Revision:	https://reviews.freebsd.org/D10824
2017-05-26 02:30:26 +00:00
emaste
a32ff2cabf bsdgrep: don't allow negative -A / -B / -C
Previously, when given a negative -A/-B/-C argument bsdgrep would
overflow the respective context flag(s) and exhibited surprising
behavior.

Fix this by removing unsignedness of Aflag/Bflag and erroring out if
we're given a value < 0.  Also adjust the type used to track 'tail'
context in procfile() so that it accurately reflects the Aflag value
rather than overflowing and losing trailing context.

This also fixes an inconsistency previously existing between -n and
-C "n" behavior.  They are now both limited to LLONG_MAX, to be
consistent.

Add some test cases to make sure grep errors out properly for both
negative context values as well as non-numeric context values rather
than giving bogus matches.

Submitted by:	Kyle Evans <kevans91@ksu.edu>
Reviewed by:	cem
Differential Revision:	https://reviews.freebsd.org/D10675
2017-05-15 17:51:01 +00:00
emaste
2b92fcdf22 bsdgrep: avoid use of magic number for REG_NOSPEC
Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Differential Revision:	https://reviews.freebsd.org/D10420
2017-05-02 21:08:38 +00:00
emaste
e028666ef0 bsdgrep: fix -w flag matching with an empty pattern
-w flag matching with an empty pattern was generally 'broken', allowing
matches to occur on any line whether or not it actually matches -w
criteria.

This fix required a good amount of refactoring to address.  procline()
is altered to *only* process the line and return whether it was a match
or not, necessary to be able to short-circuit the whole function in case
of this matchall flag. -m flag handling is moved out as well because it
suffers from the same fate as context handling if we bypass any actual
pattern matching.

The matching context (matches, mostly) didn't previously exist outside
of procline(), so we go ahead and create context object for file
processing bits to pass around.  grep_printline() was created due to
this, for the scenarios where the matches don't actually matter and we
just want to print a line or two, a la flushing the context queue and
no -o or --color specified.

Damage from this broken behavior would have been mitigated by the fact
that it is unlikely users would invoke grep -w with an empty pattern.

This was identified while checking PR 105221 for problems it this may
cause in BSD grep, but PR 105221 is *not* a report of this behavior.

Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Differential Revision:	https://reviews.freebsd.org/D10433
2017-05-02 20:39:33 +00:00
emaste
09cf7d98e5 bsdgrep: Add GNU compatible version string indicator
As reported in r218614 it's useful to have an indication of whether or not
BSD grep was built with GNU_GREP_COMPAT.

Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Reported by:	mandree
Differential Revision:	https://reviews.freebsd.org/D10451
2017-05-02 13:38:20 +00:00
emaste
dd5e4bc8dd bsdgrep: add BSD_GREP_FASTMATCH knob for built-in fastmatch
Bugs have been found in the fastmatch implementation as used in bsdgrep.
Some have been fixed (r316495) while fixes for others are in review
(D10098).

In comparison with the fastmatch implementation, Kyle Evans found that:

- regex(3)'s performance with literal expressions offers a speed
  improvement over fastmatch

- regex(3)'s performance, both with simple BREs and EREs, seems to be
  comparable

The regex implementation was imported in r226035, and the commit message
reports:

    This is a temporary solution until the whole regex library is
    not replaced so that BSD grep development can continue and the
    backported code gets some review and testing. This change only
    improves scalability slightly, there is no big performance boost
    yet but several minor bugs have been found and fixed.

Introduce a WITH_/WITHOUT_BSD_GREP_FASTMATCH knob to support testing
of both approaches.

PR:		175314, 194823
Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Reviewed by:	bdrewery (in part)
Differential Revision:	https://reviews.freebsd.org/D10282
2017-04-21 14:36:09 +00:00
emaste
a1fc8db7c1 bsdgrep: for -r, use the working directory if none specified
This is more sensible than the previous behaviour of grepping stdin,
and matches newer GNU grep behaviour.

PR:		216307
Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Reviewed by:	cem, emaste, ngie
Relnotes:	Yes
Differential Revision:	https://reviews.freebsd.org/
2017-04-17 13:22:39 +00:00
emaste
cf9dd707cc bsdgrep: add -z/--null-data support
-z treats input and output data as sequences of lines terminated by a
zero byte instead of a newline. This brings it more in line with GNU grep
and brings us closer to passing the current tests with BSD grep.

Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Reviewed by:	cem
Relnotes:	Yes
Differential Revision:	https://reviews.freebsd.org/D10101
2017-04-17 13:14:18 +00:00
emaste
22cd511e9b bsdgrep: Handle special case of single-byte NUL pattern
PR:		202022
Submitted by:	Kyle Evans <kevans91 at ksu.edu>
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D10102
2017-04-05 20:11:05 +00:00
emaste
631cc497f4 bsdgrep: treat rgrep as grep -r
Submitted by:	   Kyle Evans <kevans91@ksu.edu>
2017-04-03 21:10:02 +00:00
bapt
4ecca13b4c Remove usage of _WITH_GETLINE from usr.bin 2016-07-30 01:07:47 +00:00
pfg
f0d9946f04 bsdgrep: fix regression in the -f option since r268799
Caused by an incomplete merge from NetBSD.

PR:		198725
MFC after:	3 days
2015-03-21 00:21:30 +00:00
pfg
642f571df9 grep: fix some memory leaks.
Bring a couple of changes from NetBSD:

queue.c (CVS Rev. 1.4. 1.5)
Fix memory leaks.
NULL does not need a cast.

grep.c (CVS Rev. 1.6)
Use the more portable getline.

Obtained from:	NetBSD
MFC after:	3 days
2014-07-17 15:48:11 +00:00
pfg
2a89efd87b Various style(9) fixes and typos in grep, sort and patch.
MFC after:	3 days
2014-04-21 22:52:18 +00:00
gabor
e072d47ed3 - Avoid unnecessary strdup()
Submitted by:	ache
MFC after:	5 days
2013-01-20 11:58:49 +00:00
gabor
037843a122 - Fix handling of the case when multiple patterns are specified in a single
command line argument, separated by newlines

PR:		bin/173673
Submitted by:	ache
MFC after:	1 week
2013-01-05 14:52:31 +00:00
eadler
7e7d24dbbf Make bsdgrep behave as gnugrep and as documented: -m should only stop
reading the specific file, not any file.

Tested by:	frogs (irc)
Reviewed by:	gabor
Approved by:	cperciva (implicit)
MFC after:	1 week
2012-12-20 17:38:14 +00:00
ed
af31e8843c More -Wmissing-variable-declarations fixes.
In addition to adding `static' where possible:

- bin/date: Move `retval' into extern.h to make it visible to date.c.
- bin/ed: Move globally used variables into ed.h.
- sbin/camcontrol: Move `verbose' into camcontrol.h and fix shadow warnings.
- usr.bin/calendar: Remove unneeded variables.
- usr.bin/chat: Make `line' local instead of global.
- usr.bin/elfdump: Comment out unneeded function.
- usr.bin/rlogin: Use _Noreturn instead of __dead2.
- usr.bin/tset: Pull `Ospeed' into extern.h.
- usr.sbin/mfiutil: Put global variables in mfiutil.h.
- usr.sbin/pkg: Remove unused `os_corres'.
- usr.sbin/quotaon, usr.sbin/repquota: Remove unused `qfname'.
2012-10-19 14:49:42 +00:00
eadler
0c3c0d072d Remove duplicate line from usage
PR:		bin/164139
Submitted by:	Yuri Pankov <yuri.pankov@gmail.com>
Approved by:	nwhitehorn
MFC after:	3 days
2012-01-15 17:01:28 +00:00
ed
2547a674dd Add missing "static const" to long options table.
This table is only used in this C file and passed to getopt_long(), so
we can safely add static and const to it.
2011-12-10 18:21:03 +00:00
gabor
ecc4a991f3 - Match GNU behavior of exit code
- Rename variable that has a different meaning now

PR:		bin/162930
Submitted by:	Jan Beich <jbeich@tormail.net>
MFC after:	1 week
2011-12-07 12:25:28 +00:00
gabor
afe335edb5 - Use getprogname() instead of __progname
- Allow disabling bzip2 support with WITHOUT_BZIP2
- Fix handling patterns that start with a dot
- Remove superfluous semicolon

Approved by:	delphij (mentor)
2011-10-11 22:27:23 +00:00
gabor
1cb16d9887 Update BSD grep to the latest development version. It has some code
backported that was written for the TRE integration project in Google
Summer of Code 2011.  This is a temporary solution until the whole
regex library is not replaced so that BSD grep development can continue
and the backported code gets some review and testing.  This change only
improves scalability slightly, there is no big performance boost yet
but several minor bugs have been found and fixed.

Approved by:	delphij (mentor)
Sposored by:	Google Summer of Code 2011
MFC after:	1 week
2011-10-05 09:56:43 +00:00
gabor
64a3d10a38 - Fix handling of environmental variables when they are set to empty string
Submitted by:	ttsestt@gmail.com
Approved by:	re (kib), delphij (mentor)
2011-08-17 13:56:33 +00:00
gabor
3b40cce52c - Use REG_NOSUB to bypass submatch counting when not necessary. This may
yield in somewhat better performance in a few cases.

Approved by:	delphij (mentor)
2011-06-12 12:51:58 +00:00
gabor
43c556e7f5 - Adjust a comment to actual behaviour
- Makefile nit
- Add more CVS/SVN keywords to make it easier to track changes from NetBSD
  in case they add further improvements

Approved by:	delphij (mentor)
Obtained from:	The NetBSD Project
2011-04-07 13:03:35 +00:00
delphij
1c0b6c81ef Revert a minor part of revision 211364:
- Imply -h if single file is grepped, this is the GNU behaviour

This is already done by code above the change and have caused a regression
since this instance of code does not check Hflag.

Reported by:	davidxu
Pointy hat to:	delphij
2010-08-19 22:55:17 +00:00
des
2dc227ede0 UTFize my name. 2010-08-19 09:28:59 +00:00
gabor
5ee11c8783 - Revert strlcpy() changes to memcpy() because it's more efficient and
former may be safer but in this case it doesn't add extra
  safety [1]
- Fix -w option [2]
- Fix handling of GREP_OPTIONS [3]
- Fix --line-buffered
- Make stdin input imply --line-buffered so that tail -f can be piped
  to grep [4]
- Imply -h if single file is grepped, this is the GNU behaviour
- Reduce locking overhead to gain some more performance [5]
- Inline some functions to help the compiler better optimize the code
- Use shortcut for empty files [6]

PR:		bin/149425 [6]
Prodded by:	jilles [1]
Reported by:	Alex Kozlov <spam@rm-rf.kiev.ua> [2] [3],
		swell.k@gmail.com [2],
		poyopoyo@puripuri.plala.or.jp [4]
Submitted by:	scf [5],
		Shuichi KITAGUCHI <ki@hh.iij4u.or.jp> [6]
Approved by:	delphij (mentor)
2010-08-15 22:15:04 +00:00
gabor
32773dc6ee - Some minor changes to the messages to increase usefulness of error msgs
Reviewed by:	hrs (Japanese catalogs),
		pluknet <pluknet at gmail dot com> (Russian catalog)
Approved by:	delphij (mentor)
2010-07-29 18:02:57 +00:00
gabor
d3f9c5a1b7 - Use the traditional behaviour for filename and directory name inclusion
and exclusion patterns [1]
- Some improvements on the exiting code, like replacing memcpy with
  strlcpy/strcpy

Approved by:	delphij (mentor)
Pointed out by:	bf [1], des [1]
2010-07-29 00:11:14 +00:00
gabor
6a65a7e10a - Fix --color behaviour to only output color sequences if stdout is a tty
or if forced mode is specified [1]
- While here, add some alternative names for the options and make then
  case-insensitive
- Fix -q and -l behaviour [2]
- Some small changes to make the code easier to review

Submitted by:   swell.k@gmail.com [1],
                dougb [2]
Approved by:    delphij (mentor)
2010-07-25 08:42:18 +00:00
delphij
aa17bd87c6 Fix crashes when using grep -R:
- Explicitly pre-zero memory for fts_open parameters.
 - Don't test against directory patterns when we are testing direct
   leaf of current directory.

While I'm there plug a few of memory leaks.
2010-07-23 19:36:11 +00:00
gabor
17349bffe4 Add BSD grep to the base system and make it our default grep.
Deliverables: Small and clean code (1,4 KSLOC vs GNU's 8,5 KSLOC),
              lower memory usage than GNU grep, GNU compatibility,
              BSD license.

TODO:         Performance is somewhat behind GNU grep but it is only
              significant for bigger searches.  The reason is complex, the
              most important factor is that GNU grep uses lots of
              optimizations to improve the speed of the regex library.
              First, we need a modern regex library (practically by adopting
              TRE), add support for GNU-style non-standard regexes and then
              reevalute the performance issues and look for bottlenecks.  In
              the meantime, for those, who need better performance, it is
              possible to build GNU grep by setting WITH_GNU_GREP.

Approved by:            delphij (mentor)
Obtained from:          OpenBSD (http://www.openbsd.org/cgi-bin/cvsweb/src/usr.bin/grep/),
                        freegrep (http://github.com/howardjp/freegrep)
Sponsored by:           Google SoC 2008
Portbuild tests run by: kris, pav, erwin
Acknowledgements to:    fjoe (as SoC 2008 mentor),
                        everyone who helped in reviewing and testing
2010-07-22 19:11:57 +00:00