Commit Graph

244 Commits

Author SHA1 Message Date
Kyle Evans
98369a6980 sed: fix hex_subst test after after r363679
r363679 is in-fact the future change referenced by the comment, helpfully
left and forgotten by kevans. Instead of just silently not matching, we
should now be erroring out with vigor.
2020-07-30 13:36:24 +00:00
Yuri Pankov
14fdf16371 sed: treat '[' as ordinary character in 'y' command
'y' does not handle bracket expressions, treat '[' as ordinary character
and do not apply bracket expression checks (GNU sed agrees).

PR:		247931
Reviewed by:	pfg, kevans
Tested by:	antoine (exp-run), Quentin L'Hours <lhoursquentin@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D25640
2020-07-26 09:15:05 +00:00
Yuri Pankov
e4bfd2a675 sed/multi_test: print comment for current test, not next one
This is visible when e.g. 8.20 is failing:

not ok 96 8.20 # \ in y command

8.20's comment should be '[ as an s delimiter and its escapes'.
2020-07-13 01:32:37 +00:00
Mateusz Piotrowski
c6f7593c1d Remove some more duplicate test cases I accidentally committed
Reported by:	markj, yuripv
MFC after:	2 weeks
X-MFC-With:	362017
2020-06-11 18:14:34 +00:00
Mateusz Piotrowski
df471ef449 Remove duplicate lines from sed tests
Reported by:	yuripv
Approved by:	pfg (src)
MFC after:	2 weeks
X-MFC-With:	362017
2020-06-11 03:03:52 +00:00
Mateusz Piotrowski
f9ab72bb47 Read commands from stdin when -f - is passed to sed(1)
This patch teaches sed to interpret a "-" in a special way when given
as an argument to the -f flag.

This behavior is also present in GNU sed.

PR:		244872
Tested by:	antoine (exp-run)
Reviewed by:	pfg, tobik (older version)
Approved by:	pfg (src)
Relnotes:	yes
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D24079
2020-06-10 19:23:58 +00:00
Kyle Evans
3a4d70c588 sed: attempt to learn about hex escapes (e.g. \x27)
Somewhat predictably, software often wants to use \x27/\x24 among others so
that they can decline worrying about ugly escaping, if said escaping is even
possible. Right now, this software is using these and getting the wrong
results, as we'll interpret those as x27 and x24 respectively. Some examples
of this, when an exp-run was ran, were science/octopus and misc/vifm.

Go ahead and process these at all times.  We allow either one or two digits,
and the tests account for both.  If extra digits are specified, e.g. \x2727,
then the third and fourth digits are interpreted literally as one might
expect.

PR:		229925
MFC after:	2 weeks
2020-06-07 04:32:38 +00:00
Benedict Reuschling
8655a965fd Update SYNOPSIS section to be consistent regarding -u, -i, and -I.
Apparently, when the -u, -i and -I options where added to sed(1), it was
forgotten to add them to both lines in the SYNOPSIS section. They were only
added to the second line, although they apply to both.

With the updated SYNOPSIS, it is now allowed (and consistent) to run:

sed -i BAK s/foo/bar/g some_file

PR:		240556
Submitted by:	Oliver Fromme
MFC after:	5 days
2020-05-19 09:17:20 +00:00
Kyle Evans
6e816d8711 sed: process \r, \n, and \t
This is both reasonable and a common GNUism that a lot of ported software
expects.

Universally process \r, \n, and \t into carriage return, newline, and tab
respectively. Newline still doesn't function in contexts where it can't
(e.g. BRE), but we process it anyways rather than passing
UB \n (escaped ordinary) through to the underlying regex engine.

Adding a --posix flag to disable these was considered, but sed.1 already
declares this version of sed a super-set of POSIX specification and this
behavior is the most likely expected when one attempts to use one of these
escape sequences in pattern space.

This differs from pre-r197362 behavior in that we now honor the three
arguably most common escape sequences used with sed(1) and we do so outside
of character classes, too.

Other escape sequences, like \s and \S, will come later when GNU extensions
are added to libregex; sed will likely link against libregex by default,
since the GNU extensions tend to be fairly un-intrusive.

PR:		229925
Reviewed by:	bapt, emaste, pfg
Differential Revision:	https://reviews.freebsd.org/D22750
2019-12-10 19:16:00 +00:00
Emmanuel Vadot
a7b5a3d486 pkgbase: Put a lot of binaries and lib in FreeBSD-runtime
All of them are needed to be able to boot to single user and be able
to repair a existing FreeBSD installation so put them directly into
FreeBSD-runtime.

Reviewed by:    bapt, gjb
Differential Revision:  https://reviews.freebsd.org/D21503
2019-09-05 14:13:08 +00:00
Yuri Pankov
d9e8d39f8d usr.bin/sed/tests: fix one of the regression test cases by adding its
results file to the build.

Reviewed by:	kevans
Approved by:	kib (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17776
2018-10-31 15:04:41 +00:00
Mark Johnston
b480502648 sed: Fix -i option behavior with 'q' command.
Don't just exit when encountering the 'q' command if we edit file
inplace, and give mf_fgets() a chance to actually handle the
inplace case.

Also add a regression test.

Submitted by:	Yuri Pankov <yuripv@yuripv.net>
Approved by:	re (kib)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D16798
2018-08-29 17:09:03 +00:00
Pedro F. Giffuni
43daed4774 Revert r337419.
The fix is only partial and causes an asymmetry which breaks a test in
multi_test.sh.

We should consider both parts of the issue found in OpenBSD[1], but for now
just revert the change.

[1] http://undeadly.org/cgi?action=article;sid=20180728110010

Reported by: asomers
2018-08-16 18:35:39 +00:00
Pedro F. Giffuni
5102499601 sed(1): partial fix for the case of the regex delimited with '['.
We don't generally support the weird case of regular expresions delimited
by an opening square bracket ('[') but POSIX says that inside
bracket expressions, escaping is not possible and both '[' and '\'
represent themselves.

PR:		230198 (exp-run)
Obtained from:	OpenBSD
2018-08-07 14:47:39 +00:00
Pedro F. Giffuni
1a68918570 sed: compile with the default WARNS level. 2018-07-30 19:41:54 +00:00
Pedro F. Giffuni
0649b6706f sed: unsign some indexes to fix sign-compare warnings.
Hinted by:	OpenBSD (CVS 1.32)

MFC after:	 1 week
2018-07-30 18:29:46 +00:00
Pedro F. Giffuni
51a8f735f8 sed(1): Suppress implicit-fallthrough.
Apparently some tools are not able to determine if all the cases of a
switch are covered. Make use of the attribute for cases like this.

Hinted by:	DragonFlyBSD GGC8 (but fixed differently)

CID:	976552
2018-07-09 20:27:31 +00:00
Pedro F. Giffuni
8a16b7a18f General further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:49:47 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Enji Cooper
d511b20a69 Add HAS_TESTS to all Makefiles that are currently using the
`SUBDIR.${MK_TESTS}+= tests` idiom.

This is a follow up to r321912.
2017-08-02 08:50:42 +00:00
Enji Cooper
4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Benedict Reuschling
0ae98ab709 Fix the last example to really replace all occurrences of the search string
in the file by adding the global (g) option at the end.  Without it, only the
first match is replaced, subsequent ones are ignored.  The intention of the
example is to demonstrate something else, but adding the g matches the example
to what the description says.

Discussed with:	    brd (on IRC)
MFC after:	    1 week
2017-04-24 16:52:26 +00:00
Brad Davis
d09b527f96 Add some useful examples to the sed man page.
Reviewed by:	wblock, bcr
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D9958
2017-03-27 16:01:44 +00:00
Warner Losh
fbbd9655e5 Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
Enji Cooper
e58164041c Don't expect :inplace_symlink_src to fail anymore (post-r313277)
The S_ISREG check was restored, such that the code will again fail with
in-place replacements on symlinks

MFC after:	12 days
X-MFC with:	r313277
Sponsored by:	Dell EMC Isilon
2017-02-07 18:23:44 +00:00
Xin LI
8e2106f0cb Restore r312404:
Use S_ISREG instead of manual & (also it's better to compare the
result from & and the pattern instead of just assuming it's one bit
value).

Pointed out by Tianjie Mao <tjmao tjmao com>.

MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D4827
2017-02-05 08:51:41 +00:00
Enji Cooper
68730a222d Add testcases for -i with hardlinks and symlinks
The symlink testcase is expected to fail, post-r312519 (the revert of
r312404); mark it so.

MFC after:	3 weeks
Sponsored by:	Dell EMC Isilon
2017-01-20 07:11:21 +00:00
Enji Cooper
58b107a311 Integrate contrib/netbsd-tests/usr.bin/sed/t_sed.sh into the FreeBSD test
suite as usr.bin/sed/sed_test

Don't expect :emptybackref to fail -- it succeeds on FreeBSD

MFC after:	3 weeks
Sponsored by:	Dell EMC Isilon
2017-01-20 06:47:02 +00:00
Xin LI
1bcffc747f Revert r312404 as we need to do an exp-run and fix existing ports that
rely on the previous behavior.

Requested by:	antonie (portmgr)
2017-01-20 06:45:06 +00:00
Xin LI
805ca1775e Use S_ISREG instead of manual & (also it's better to compare the
result from & and the pattern instead of just assuming it's one bit
value).

Pointed out by Tianjie Mao <tjmao tjmao com>.

MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D4827
2017-01-19 08:01:35 +00:00
Pedro F. Giffuni
b780b03cbe sed(1): add LEGACY_BSDSED_COMPAT compile-time flag.
In r297602, which included a __FreeBSD_version bump to 1100105, we changed
sed 'i' and 'a' from discarding whitespaces to conform with what GNU and
sysvish sed do.

There are arguments in favor of keeping the old behavior but the new
behavior is also useful for migration purposes. It seems important to at
least consider the case of developers depending on the previous behavior,
so add a CFLAG to enable the old behaviour.

PR:		213474
MFC after:	5 days
2016-11-04 20:49:59 +00:00
Pedro F. Giffuni
249a8a8003 sed(1): Revert r303047 "cleanup" and therefore r303572.
While big, the change was meant to have no effect on behavior and instead
so far we have found two regressions: one in the etcupdate tests and
another one in the games/openttd port[1].

Revert to a known working state. We will likely have to split the patch in
functional parts before bringing back the changes.

PR:		195929
Reported by:	danfe, madpilot [1]
2016-08-02 15:35:53 +00:00
Enji Cooper
637a1a7446 Fix regression with /i caused by r303047
'\n' was specifically added to -e arguments prior to r303047. Restore
historical behavior which in turn fixes usr.sbin/etcupdate/preworld_test:main .

The fix is being committed to address the issue in the short term and may be
iterated upon as noted in bug 211399

Discussed with:		mi, pfg
Differential Revision:	https://reviews.freebsd.org/D7368
PR:			195929, 211399 [*]
MFC after:		18 days
X-MFC with:		r303047
Reported by:		Jenkins
Sponsored by:		EMC / Isilon Storage Division
2016-07-31 05:31:09 +00:00
Enji Cooper
131cac80e1 Explicitly test for cu_fgets returning NULL or !NULL
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
2016-07-30 04:40:44 +00:00
Baptiste Daroussin
a43d43d574 Remove usage of _WITH_GETLINE from usr.bin 2016-07-30 01:07:47 +00:00
Ed Schouten
6718ba1851 Don't call basename() and dirname() in an unportable way.
POSIX allows these functions to modify their input buffer, so that they
have storage for the return value. Pull copies of the filename before
calling these utility functions.
2016-07-28 15:17:12 +00:00
Enji Cooper
60b0ea0dfb Testcase 7.8 no longer needs to be marked TODO
It passes out of the box today

MFC after: 2 months
Sponsored by: EMC / Isilon Storage Division
2016-07-27 06:49:16 +00:00
Pedro F. Giffuni
f05dbca7c7 sed(1): Appease older GCC. 2016-07-21 14:17:36 +00:00
Pedro F. Giffuni
b4932d18ee sed(1): Assorted cleanups and simplifications.
Const-ify several variables, make it build cleanly with WARNS level 5.

Submitted by:	mi
PR:		195929
MFC after:	1 month
2016-07-19 22:56:40 +00:00
Pedro F. Giffuni
4386fba70c sed(1): Fix off by one introduced in r299211.
Detected by running the gsed tests.

Submitted by:	Mikhail Teterin
PR:		195929
MFC after:	3 days
2016-07-17 21:49:53 +00:00
Pedro F. Giffuni
7e77774311 sed(1): Fix a mismatch and sync with the OpenBSD's commit.
This was causing some strange behaviour.

Reported by:	olivier
Obtained from:	OpenBSD (CVS rev. 1.28)
2016-06-01 16:53:02 +00:00
Pedro F. Giffuni
924b84c108 sed: convert sed to use REG_STARTEND more explicitly.
Summarizing the findings in the OpenBSD list:

This solves a reproduceable issue with very recent Mesa where REG_NOTBOL
combined with a match at the begin of the string causes our regex library
to treat the word as not begin of word.

Thanks to Martijn van Duren and Ingo Schwarze for taking the time to
solve this in the least invasive way.

PR:		209352, 209387
Taken from:     openbsd-tech (Martijn van Duren)
MFC after:	1 month
2016-05-25 15:42:39 +00:00
Pedro F. Giffuni
6351d60947 sed.1: Correction for the case insensitive case.
Use the capital I instead of the lowercase.

Submitted by:	Mikhail T.
PR:		195929
MFC after:	2 weeks
2016-05-10 17:01:51 +00:00
Pedro F. Giffuni
11398c77da Revert r299279:
Simplify redundant malloc'ing in sed -e.

It is causing havoc in the ports tree:

===>  Configuring for wxsvg-1.5.7
sed: 1: "/gcc_dir=\\`/s/gcc /$CC /": bad flag in substitute command: '/'
*** Error code 1

===>  Patching for vips-8.3.1
sed: 1: "1s|^#![[:space:]]*/usr/ ...": bad flag in substitute command: 's'
*** Error code 1

PR:		195929
Reported by:	danilo
2016-05-10 02:02:50 +00:00
Pedro F. Giffuni
d74b9e1311 Simplify redundant malloc'ing in sed -e.
When encountering an -e argument, sed currently mallocs a string to COPY
the optarg -- with '\n' appended. The appendage does not seem necessary --
indeed, the same call to add_compunit processing the sole command (given
without -e) passes the *argv verbatim: without making a copy, and without
appending newline.

This matches what is done in other BSDs.

Submitted by:	Mikhail T.
PR:		195929
MFC after:	2 weeks
2016-05-09 18:53:46 +00:00
Pedro F. Giffuni
b6f5aa5715 sed: rewrite the main loop.
Rewrite the main loop of the "sed s/..." command, shortening it by ten
lines and simplifying it by removing the switch statement implementing
/g, /1, and /2 separately and repetitively.

This will be needed to bring a fix from OpenBSD later.

Obtained from:	OpenBSD (schwarze CVS Rev. 1.18)
MFC after:	3 weeks
2016-05-07 01:44:22 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Pedro F. Giffuni
e55a6575d9 kernel: use our nitems() macro when it is available through param.h.
No functional change, only trivial cases are done in this sweep,

Discussed in:	freebsd-current
2016-04-21 21:30:51 +00:00
Glen Barber
2263fb580e MFH
Sponsored by:	The FreeBSD Foundation
2016-04-06 01:44:21 +00:00
Pedro F. Giffuni
678fec509d Fix sed functions 'i' and 'a' from discarding leading white space.
This appears to be implementation dependent but convenient and makes
our sed behave more like GNU sed.

Given that it is not the historic behavior, bump FreeBSD_version
should userland/ports somehow depend on it.

Obtained from:	NetBSD (bin/49872)

Reviewed by:	bdrewery
PR:		208554
Merge after:	NEVER
2016-04-06 00:55:39 +00:00