64 Commits

Author SHA1 Message Date
jilles
a90ad707c5 sh: Fix use-after-free if a trap replaces itself.
MFC after:	1 week
2016-03-28 18:58:40 +00:00
bdrewery
aab40fdc3d DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
jilles
6e5195ef20 sh: Add test for 'set -o nolog'.
The option does not do anything so check that the output of 'set +o' is
different.
2016-03-09 21:05:21 +00:00
jilles
c7d6350d16 sh: Fix set -v and multiline history after r295825.
r295825 erroneously dropped the newline from the input data for 'set -v'
output and multiline history.

Reported by:	vangyzen
2016-02-23 22:44:01 +00:00
jilles
5e2165399a sh: Rework code to remove '\0' from shell input.
This fixes bugs where '\0' was not removed correctly and speeds up the
parser.
2016-02-19 21:53:12 +00:00
jilles
f4563d3450 sh: Add tests for comments in sh -c. 2016-02-19 16:56:07 +00:00
bdrewery
002a527f3e Test directories can build in parallel fine.
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 02:13:59 +00:00
jilles
e81a328bbd sh: Clean a readonly local, even if the variable does not exist outside.
If a local variable has been made read-only, this should not prevent its
removal when the function returns.
2016-01-22 20:10:08 +00:00
jilles
9d436f6b51 sh: Add already working test for local-readonly interaction. 2016-01-22 18:10:36 +00:00
jilles
d42a26ab20 sh: Update associated state when restoring locals while leaving a function.
Some variables like PATH call a function when modified. Make sure to call
this also when leaving a function where such a variable was made local.

Make sure to restore local variables before shellparam, so getopts state is
not clobbered.
2016-01-10 16:31:28 +00:00
jilles
2d4b37e81d sh: Add a test for 'cd -'.
Redirect 'cd -' output to /dev/null since POSIX requires it to write the new
directory name even if not interactive, but we currently only write it if
interactive.
2016-01-07 21:46:07 +00:00
jilles
68a5d0c372 sh: Ensure OPTIND=1 in subshell without forking does not affect outer env.
Command substitutions containing a single simple command and here-document
expansion are performed in a subshell environment, but may not fork. Any
modified state of the shell environment should be restored afterward.

The state that OPTIND=1 had been done was not saved and restored here.

Note that the other parts of shellparam need not be saved and restored,
since they are not modified in these situations (a fork is done before such
modifications).
2016-01-07 20:48:24 +00:00
jilles
99982ba080 sh: Link tests/builtins/getopts9.0 to the build.
This was forgotten in r273700.
2016-01-03 22:16:27 +00:00
jilles
a11e57aacc sh: Add tests for #/##/%/%% on $* and $@.
Although POSIX leaves things like ${*#X} unspecified, it occasionally occurs
in practice. Add some tests that seem to work sensibly.
2015-12-26 22:27:48 +00:00
jilles
bfe5e9c164 sh: Add limited test for ${#@} and ${#*}.
POSIX leaves the result of expanding ${#@} and ${#*} unspecified, but ensure
it is numeric.
2015-12-06 14:09:31 +00:00
jilles
dfc5b1f4e5 sh: Link tests/parameters/positional8.0 to the build.
This was forgotten in r291025.
2015-12-05 12:55:34 +00:00
jilles
cf0d26dcd6 sh: Fix ""$@, which should not use the special case for "$@".
"$@" should expand to no words if there are no positional parameters, but
""$@ should always expand to at least an empty word.
2015-11-18 21:09:03 +00:00
bdrewery
153205e33f Add more SUBDIR_PARALLEL.
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 22:55:08 +00:00
ngie
dd1f618367 Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
2015-10-12 08:16:03 +00:00
jilles
6633d3a788 sh: Allow empty << EOF markers. 2015-09-02 19:49:55 +00:00
jilles
1fb3e6c2bb sh: Fix read with escaped IFS characters at the end.
Characters escaped with a backslash must be treated as if they were not in
IFS. This includes stripping trailing IFS characters.
2015-08-30 17:24:22 +00:00
jilles
424480153f sh: Fix out of bounds read when there is no ] after a [:class:].
The initial check for a matching ] was incorrect if a ] may be consumed by a
[:class:]. The subsequent loop assumed that there must be a ].

Remove the initial check and make the loop cope with a missing ].

Found with afl-fuzz.

MFC after:	1 week
2015-08-25 21:55:15 +00:00
jilles
f504ca457f sh: Don't create bad parse result when postponing a bad substitution error.
An invalid substitution like ${var@} does not cause a parse error but is
stored in the intermediate representation, to be written as part of the
error message. If there is a CTL* byte in the stored part, this confuses
some code such as the code to skip an unused alternative such as in
${var-alternative}.

To keep things simple, do not store CTL* bytes.

Found with afl-fuzz.

MFC after:	1 week
2015-08-23 20:44:53 +00:00
jilles
5e4f40c860 sh: Avoid negative character values from $'\Uffffffff' etc.
The negative value was not expected and generated the low 8 bits as a byte,
which may be an invalid character encoding.

The final shift in creating the negative value was undefined as well.

Make the temporary variable unsigned to fix this.
2015-08-20 22:05:55 +00:00
jilles
5d7936e951 sh: Add test for $'\u' without any digits.
It is likely that $'\uXXXX' and $'\UXXXXXXXX' will allow fewer digits in
future. However, no digits at all should still be disallowed.
2015-08-20 21:31:36 +00:00
jilles
718cf25701 sh: Backslash-newline should not affect field splitting in read builtin.
This was originally broken in r212339 in 2010.
2015-08-16 12:57:17 +00:00
bdrewery
5ae6ee36a9 sh: Fix the trap builtin to be POSIX-compliant for 'trap exit SIG' and 'trap n n...'.
The parser considered 'trap exit INT' to reset the default for both EXIT and
INT. This beahvior is not POSIX compliant. This was avoided if a value was
specified for 'exit', but then disallows exiting with the signal received. A
possible workaround is using ' exit'.

However POSIX does allow this type of behavior if the parameters are all
integers. Fix the handling for this and clarify its support in the manpage
since it is specifically allowed by POSIX.

Differential Revision:	https://reviews.freebsd.org/D2325
Reviewed by:	jilles
MFC after:	2 weeks
2015-04-18 23:49:57 +00:00
jilles
1a2e63b4e8 sh: Add more tests for exotic IFS splitting. 2015-03-31 20:59:37 +00:00
jilles
8f5603742f sh: Add simple tests for globbing/splitting command substitution. 2015-02-15 19:48:29 +00:00
jilles
412e06f616 sh: Make sure output suitable as shell input is also printable.
Commands like 'export -p', 'set' and 'trap', and tracing enabled via 'set
-x' generate output suitable as shell input by adding quotes as necessary.

If there are control characters other than newline or invalid UTF-8
sequences, use $'...' and \OOO to display them safely.

The resulting output is not parsable by a strict POSIX.1-2008 shell but sh
from FreeBSD 9.0 and newer and many other shells can parse it.
2014-12-14 16:26:19 +00:00
jilles
a2ae3a60f9 sh: Fix corruption of CTL* bytes in positional parameters in redirection.
EXP_REDIR was not being checked for while expanding positional parameters in
redirection, so CTL* bytes were not being prefixed where they should be.

MFC after:	1 week
2014-10-31 22:28:10 +00:00
jilles
10a925c0a7 Treat IFS separators in "$*" as quoted.
This makes a difference if IFS starts with *, ?, [ or a CTL* byte.
2014-10-28 22:14:31 +00:00
jilles
bc5702addf sh: Make getopts memory-safe if with changing arguments.
POSIX does not permit to continuing a getopts loop with different
arguments. For parsing the positional parameters, we handle this case by
resetting the getopts state when the positional parameters are changed in
any way (and the getopts state is local to a function). However, in the
syntax getopts <optstring> <var> <arg...>, changes could lead to invalid
memory access.

In the syntax getopts <optstring> <var> <arg...>, store a copy of the
arguments and continue to use them until getopts is reset.
2014-10-26 17:50:33 +00:00
jilles
4eb9d5ad9f sh: Allow backslash-newline continuation in more places:
* directly after a $
 * directly after ${
 * between the characters of a multi-character operator token
 * within a parameter name
2014-10-19 11:59:15 +00:00
jilles
a6283e5687 sh: Add some tests for backslash-newline continuation. 2014-10-17 21:52:57 +00:00
jilles
2791f385b9 sh: Fix break/continue/return in multiline eval.
Example:
  eval $'return\necho bad'
2014-10-12 13:12:06 +00:00
jilles
ce43ba6175 sh: Fix LINENO and prompt after $'\0 and newline. 2014-10-03 20:24:56 +00:00
jilles
b9a6c69366 sh: Remove arbitrary length limit on << EOF markers.
This also simplifies the code.
2014-09-14 16:46:30 +00:00
jilles
7f580421d9 sh: Don't hardcode relative paths in the tests stderr files.
These paths have had to be adjusted to changes in the testsuite runner
several times, so modify the tests to remove the need for such adjustment.

A cp in functional_test.sh is now unneeded, but this matters little in
performance.
2014-08-17 14:26:12 +00:00
jilles
948728c4a6 sh: Mask off shift distance (<< and >>) in arithmetic.
In C, shift distances equal to or larger than the number of bits in the
operand result in undefined behaviour. As part of eliminating undefined
behaviour in arithmetic, mask off the distance like Java and JavaScript
specify and C on x86 usually does.

Assumption: conversion from unsigned to signed retains the two's complement
bits.
Assumption: uintmax_t has no padding bits.
2014-08-15 22:36:41 +00:00
ngie
45e6755fc1 Convert bin/sh/tests to ATF
The new code uses a "test discovery mechanism" to determine
what tests are available for execution

The test shell can be specified via:

  kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh

Sponsored by: EMC / Isilon Storage Division
Approved by: jmmv (mentor)
Reviewed by: jilles (maintainer)
2014-08-13 04:14:50 +00:00
jilles
8c44f4d5e2 sh: Allow arbitrarily large numbers in break and continue.
The argument is capped to loopnest, so strtol()'s [ERANGE] can be ignored.
2014-07-20 20:29:09 +00:00
jilles
9af633c667 sh: Correctly handle positional parameters beyond INT_MAX on 64-bit systems.
Currently, there can be no more than INT_MAX positional parameters. Make
sure to treat all higher ones as unset to avoid incorrect results and
crashes.

On 64-bit systems, our atoi() takes the low 32 bits of the strtol() and
sign-extends them.

On 32-bit systems, the call to atoi() returned INT_MAX for too high values
and there is not enough address space for so many positional parameters, so
there was no issue.
2014-07-12 21:54:11 +00:00
jilles
16670f2045 sh: Consistently treat ${01} like $1.
Leading zeroes were ignored when checking whether a positional parameter is
set, but not when expanding its value. Ignore leading zeroes in any case.
2014-07-12 10:27:30 +00:00
jilles
26d6f04885 sh: Add test for ${01} and ${010} that already works.
Although it is probably unwise to use this, POSIX is clear that leading
zeroes are permitted in positional parameters (and do not indicate octal).

Such positional parameters are checked for being unset and/or null
correctly, but their value is incorrectly expanded.
2014-07-08 22:04:44 +00:00
jilles
6b378329a2 Don't install locale1.0 if MK_NLS == no.
The test locale1.0 depends on locale support; it is meaningless without a
working LC_MESSAGES.

I added an OptionalObsoleteFiles.inc entry.

PR:		181151
Submitted by:	Garrett Cooper (original version)
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-07-08 21:50:13 +00:00
jilles
aa09279cea sh: In getopts, unset OPTARG where POSIX says we should. 2014-05-10 19:18:49 +00:00
jilles
469deb4b78 sh: Add new tests to the Makefile. 2014-05-10 19:06:36 +00:00
jilles
56ecdb9ab8 sh: Don't discard getopts state on unknown option or missing argument.
When getopts finds an invalid option or a missing option-argument, it should
not reset its state and should set OPTIND as normal. This is an old ash bug
that was fixed long ago in dash. Our behaviour now matches most other
shells.
2014-05-10 17:42:21 +00:00
jilles
0d127c3d77 sh: Send getopts error messages to stderr, not stdout.
Adjust a testcase for this change.
2014-05-09 13:32:36 +00:00