Commit Graph

1221 Commits

Author SHA1 Message Date
Glen Barber
43faedc133 First pass to fix the 'tests' packages.
Sponsored by:	The FreeBSD Foundation
2016-02-02 22:26:49 +00:00
Jilles Tjoelker
bd9b38d1f7 sh: Don't allocate a redirtab if there are no redirections.
Builtins (including variable assignments without command word), function
calls and redirected compound commands need to restore file descriptors
to their original state after execution. This is handled by allocating a
redirtab structure. These mallocs and frees show up heavily in pmcstat.

Only allocate a redirtab if there are actually redirections and maintain a
count of how many levels of REDIR_PUSH there are without redirtabs.

A simple loop without external programs like

sh -c 'i=0; w=$(printf %0100d 7); while [ "$i" -lt 1000000 ]; do
    i=$((i+1)); done'

is over 25% faster on an amd64 bhyve VM.
2016-01-30 21:21:25 +00:00
Jilles Tjoelker
7b6779b66b sh(1): Document 'cd -'.
This reflects the changes in r294649 and can therefore not be MFCed by
itself.
2016-01-30 20:10:20 +00:00
Jilles Tjoelker
52d5897d50 sh: Constify internal representation in expand.c.
Forbid (temporary or permanent) modifications of the strings in NARG nodes
during expansion.

Tilde expansion now needs to copy the username for the terminating '\0'.
2016-01-24 22:26:25 +00:00
Jilles Tjoelker
d5119a2a9a sh: Remove a global variable from cd.c. 2016-01-24 17:01:34 +00:00
Jilles Tjoelker
7b40c6df95 sh: Use OLDPWD shell variable for 'cd -'.
Per POSIX, 'cd -' should use the OLDPWD shell variable, not internal state.
This variable is normally exported.

Also, if OLDPWD is not set, fail 'cd -' instead of changing to the current
directory.
2016-01-23 23:00:38 +00:00
Jilles Tjoelker
a9bdd616f9 sh: Clean a readonly local, even if the variable does not exist outside.
If a local variable has been made read-only, this should not prevent its
removal when the function returns.
2016-01-22 20:10:08 +00:00
Jilles Tjoelker
cce13d6539 sh: Add already working test for local-readonly interaction. 2016-01-22 18:10:36 +00:00
Jilles Tjoelker
6b8e48f45e sh: Simplify some code related to positional parameters. 2016-01-19 22:41:26 +00:00
Jilles Tjoelker
cf45f1240d sh: Update associated state when restoring locals while leaving a function.
Some variables like PATH call a function when modified. Make sure to call
this also when leaving a function where such a variable was made local.

Make sure to restore local variables before shellparam, so getopts state is
not clobbered.
2016-01-10 16:31:28 +00:00
Jilles Tjoelker
3da40d4a6b sh: Reduce size of options table. 2016-01-07 23:13:20 +00:00
Jilles Tjoelker
0e50eefbd5 sh: Add a test for 'cd -'.
Redirect 'cd -' output to /dev/null since POSIX requires it to write the new
directory name even if not interactive, but we currently only write it if
interactive.
2016-01-07 21:46:07 +00:00
Jilles Tjoelker
adba77a62e sh: Ensure OPTIND=1 in subshell without forking does not affect outer env.
Command substitutions containing a single simple command and here-document
expansion are performed in a subshell environment, but may not fork. Any
modified state of the shell environment should be restored afterward.

The state that OPTIND=1 had been done was not saved and restored here.

Note that the other parts of shellparam need not be saved and restored,
since they are not modified in these situations (a fork is done before such
modifications).
2016-01-07 20:48:24 +00:00
Jilles Tjoelker
ac4ecc78c1 sh: Link tests/builtins/getopts9.0 to the build.
This was forgotten in r273700.
2016-01-03 22:16:27 +00:00
Jilles Tjoelker
d3fa2c78f2 sh: Reduce size of builtins table. 2016-01-03 21:30:22 +00:00
Jilles Tjoelker
bc57b4d469 sh: Remove redundant CTLQUOTEMARK checks.
With the new expand.c code, the intermediate representation passed to the
pathname generation code only contains CTLESC, not CTLQUOTEMARK.

CTLQUOTEMARK now only occurs in the text of NARG nodes (output of the
parser).
2015-12-31 20:15:57 +00:00
Jilles Tjoelker
ee97d8648c sh: Reindent expandmeta(). 2015-12-31 18:56:11 +00:00
Jilles Tjoelker
0e39c931e2 sh: Perform pathname generation during the first expansion phases.
This avoids the need to add and remove CTLESC bytes if pathname generation
will not be performed (set -f).

Side effect: the order of operations is slightly different: pathname
generation in ${$+* $(CMD)} will not see filesystem changes from CMD.
2015-12-31 18:47:54 +00:00
Jilles Tjoelker
47d8814c7f sh: Perform IFS splitting during the first expansion phases.
This simplifies the code and should be faster in some cases.

Side effect: the order of operations is different so that the value of IFS
used when IFS is modified during expansion (${IFS:=...}, ${IFS=...} or
$((...IFS=...))) may be different. Note that this order is highly unportable
between shells.
2015-12-31 17:51:15 +00:00
Jilles Tjoelker
4f2f5faa62 sh: Split subevalvar() in #/##/%/%% and =/? parts. 2015-12-29 20:51:29 +00:00
Jilles Tjoelker
cf1330bef9 sh: Add tests for #/##/%/%% on $* and $@.
Although POSIX leaves things like ${*#X} unspecified, it occasionally occurs
in practice. Add some tests that seem to work sensibly.
2015-12-26 22:27:48 +00:00
Jilles Tjoelker
cc684f839e sh/mknodes: Close files and check for errors when writing.
This is a build tool only and does not affect run time.

PR:		204951
MFC after:	1 week
2015-12-20 16:40:36 +00:00
Jilles Tjoelker
d41b2be159 sh: Fix use-after-free when attempting to modify a read-only variable.
Reported by:	bapt
MFC after:	1 week
2015-12-16 20:33:47 +00:00
Jilles Tjoelker
d253cfe3b2 sh: Add limited test for ${#@} and ${#*}.
POSIX leaves the result of expanding ${#@} and ${#*} unspecified, but ensure
it is numeric.
2015-12-06 14:09:31 +00:00
Jilles Tjoelker
fb6fa0a785 sh: Link tests/parameters/positional8.0 to the build.
This was forgotten in r291025.
2015-12-05 12:55:34 +00:00
Jilles Tjoelker
a83f6e1ad9 sh: Remove global state from nodes.c.
No functional change is intended.
2015-11-24 22:47:19 +00:00
Jilles Tjoelker
a5cb58abc8 sh: Fix ""$@, which should not use the special case for "$@".
"$@" should expand to no words if there are no positional parameters, but
""$@ should always expand to at least an empty word.
2015-11-18 21:09:03 +00:00
Jilles Tjoelker
046bfe5240 sh: Avoid copying argv for simple commands.
Add dummy entries before and after so arglist's array is directly usable as
argv.
2015-11-01 22:07:40 +00:00
Bryan Drewery
aa92269e46 Add more SUBDIR_PARALLEL.
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 22:55:08 +00:00
Enji Cooper
b2d48be1bc Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
2015-10-12 08:16:03 +00:00
Jilles Tjoelker
8ef0ae8a8f sh: Make struct arglist an array instead of a linked list.
This simplifies the code (e.g. allowing use of qsort(3) instead of a
hand-rolled mergesort) and should have better cache properties.

The waste of unused args arrays after resizes is approximately the same as
the savings from getting rid of the next pointers.

At the same time, remove a piece of global state and move some duplicated
code into a function.
2015-10-11 21:33:00 +00:00
Jilles Tjoelker
d358fa780b wordexp: Rewrite to make WRDE_NOCMD reliable.
Shell syntax is too complicated to detect command substitution and unquoted
operators reliably without implementing much of sh's parser. Therefore, have
sh do this detection.

While changing sh's support anyway, also read input from a pipe instead of
arguments to avoid {ARG_MAX} limits and improve privacy, and output count
and length using 16 instead of 8 digits.

The basic concept is:
execl("/bin/sh", "sh", "-c", "freebsd_wordexp ${1:+\"$1\"} -f "$2",
    "", flags & WRDE_NOCMD ? "-p" : "", <pipe with words>);

The WRDE_BADCHAR error is still implemented in libc. POSIX requires us to
fail strings containing unquoted braces with code WRDE_BADCHAR. Since this
is normally not a syntax error in sh, there is still a need for checking
code in libc, we_check().

The new we_check() is an optimistic check that all the characters
  <newline> | & ; < > ( ) { }
are quoted. To avoid duplicating too much sh logic, such characters are
permitted when quoting characters are seen, even if the quoting characters
may themselves be quoted. This code reports all WRDE_BADCHAR errors; bad
characters that get past it and are a syntax error in sh return WRDE_SYNTAX.

Although many implementations of WRDE_NOCMD erroneously allow some command
substitutions (and ours even documented this), there appears to be code that
relies on its security (codesearch.debian.net shows quite a few uses).
Passing untrusted data to wordexp() still exposes a denial of service
possibility and a fairly large attack surface.

Reviewed by:	wblock (man page only)
MFC after:	2 weeks
Relnotes:	yes
Security:	fixes command execution with wordexp(untrusted, WRDE_NOCMD)
2015-09-30 21:32:29 +00:00
Jilles Tjoelker
92fe71faae sh: Allow empty << EOF markers. 2015-09-02 19:49:55 +00:00
Jilles Tjoelker
f0688a48d2 sh: Fix read with escaped IFS characters at the end.
Characters escaped with a backslash must be treated as if they were not in
IFS. This includes stripping trailing IFS characters.
2015-08-30 17:24:22 +00:00
Jilles Tjoelker
62c3711632 sh: Add set -o nolog.
POSIX requires this to prevent entering function definitions in history but
this implementation does nothing except retain the option's value. In ksh88,
function definitions were usually entered in the history file, even when
they came from ~/.profile and the $ENV file, to allow displaying their
definitions.

This is also the first option that does not have a letter.
2015-08-29 19:41:47 +00:00
Jilles Tjoelker
4a4867d667 sh: Fix out of bounds read when there is no ] after a [:class:].
The initial check for a matching ] was incorrect if a ] may be consumed by a
[:class:]. The subsequent loop assumed that there must be a ].

Remove the initial check and make the loop cope with a missing ].

Found with afl-fuzz.

MFC after:	1 week
2015-08-25 21:55:15 +00:00
Jilles Tjoelker
d0b0ac182d sh: Don't create bad parse result when postponing a bad substitution error.
An invalid substitution like ${var@} does not cause a parse error but is
stored in the intermediate representation, to be written as part of the
error message. If there is a CTL* byte in the stored part, this confuses
some code such as the code to skip an unused alternative such as in
${var-alternative}.

To keep things simple, do not store CTL* bytes.

Found with afl-fuzz.

MFC after:	1 week
2015-08-23 20:44:53 +00:00
Jilles Tjoelker
5e03b81fd4 sh: Avoid negative character values from $'\Uffffffff' etc.
The negative value was not expected and generated the low 8 bits as a byte,
which may be an invalid character encoding.

The final shift in creating the negative value was undefined as well.

Make the temporary variable unsigned to fix this.
2015-08-20 22:05:55 +00:00
Jilles Tjoelker
f84460dded sh: Add test for $'\u' without any digits.
It is likely that $'\uXXXX' and $'\UXXXXXXXX' will allow fewer digits in
future. However, no digits at all should still be disallowed.
2015-08-20 21:31:36 +00:00
Jilles Tjoelker
4b6578dc8c sh: Backslash-newline should not affect field splitting in read builtin.
This was originally broken in r212339 in 2010.
2015-08-16 12:57:17 +00:00
Jilles Tjoelker
58bbcdb183 sh: When setting option via long name, don't go via letter.
Looking up the letter makes no sense and prevents adding options that only
have a long name, no letter.
2015-08-14 21:44:15 +00:00
Jilles Tjoelker
8956c4ec63 sh(1): libedit has supported multibyte encodings for a while. 2015-07-11 13:07:26 +00:00
Jilles Tjoelker
c3c85727d0 sh: Fix some arithmetic undefined behaviour.
Fix shifts of possibly negative numbers found with ubsan and avoid signed
integer overflow when hashing an extremely long command name.

MFC after:	1 week
2015-06-24 20:51:48 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Jilles Tjoelker
cad3cc1310 sh: Pass along SIGINT from a child if job control is enabled, even when not
interactive.

I added the interactive check in r208881 to be safe, but in actual use
(scripts in set -m mode) passing along SIGINT seems best.

Discussed with:	bdrewery
2015-04-25 13:34:25 +00:00
Bryan Drewery
a59f817491 sh: Fix the trap builtin to be POSIX-compliant for 'trap exit SIG' and 'trap n n...'.
The parser considered 'trap exit INT' to reset the default for both EXIT and
INT. This beahvior is not POSIX compliant. This was avoided if a value was
specified for 'exit', but then disallows exiting with the signal received. A
possible workaround is using ' exit'.

However POSIX does allow this type of behavior if the parameters are all
integers. Fix the handling for this and clarify its support in the manpage
since it is specifically allowed by POSIX.

Differential Revision:	https://reviews.freebsd.org/D2325
Reviewed by:	jilles
MFC after:	2 weeks
2015-04-18 23:49:57 +00:00
Jilles Tjoelker
739d568935 sh: Add more tests for exotic IFS splitting. 2015-03-31 20:59:37 +00:00
Jilles Tjoelker
58bdb0761c sh: Fix more compiler warnings related to variable declarations. 2015-03-03 21:21:43 +00:00
Jilles Tjoelker
22afca9b67 sh: Fix more compiler warnings. 2015-03-01 22:32:23 +00:00
Jilles Tjoelker
22ea47ec90 sh: Fix compiler warnings related to duplicate or missing declarations. 2015-03-01 21:46:55 +00:00
Nathan Whitehorn
c4b725f42a Fix unitialized variable that broke sh on PowerPC starting with r278826. 2015-02-26 20:59:18 +00:00
Jilles Tjoelker
4445af212c sh: Add details about importing the environment and initializing OPTIND. 2015-02-22 21:32:57 +00:00
Jilles Tjoelker
7034d8df04 sh: Various cleanups to expand.c:
* Remove some gotos.
* Remove unused parameter.
* Remove duplicate code.
2015-02-15 22:38:00 +00:00
Jilles Tjoelker
781bfb5a53 sh: Prefer "" to nullstr where possible. 2015-02-15 21:47:43 +00:00
Jilles Tjoelker
a4652c280b sh: Add stsavestr(), like savestr() but allocates using stalloc(). 2015-02-15 21:41:29 +00:00
Jilles Tjoelker
8c6cc7dad5 sh: Add simple tests for globbing/splitting command substitution. 2015-02-15 19:48:29 +00:00
Jilles Tjoelker
edebe56497 sh: Abort a wait builtin on any trapped signal.
This is required by POSIX.

PR:		197210
Reported by:	ache
MFC after:	2 weeks
2015-01-31 13:53:29 +00:00
Jilles Tjoelker
85052e7b56 sh: Move some code from onint() to onsig(), making onint() noreturn.
As a result, the INTON macro which is used many times generates fewer
bytes of code.
2014-12-21 23:09:59 +00:00
Jilles Tjoelker
f649ab8b15 sh: Remove EXP_REDIR.
EXP_REDIR was supposed to generate pathnames in redirection if exactly one
file matches, as permitted but not required by POSIX in interactive mode. It
is unlikely this will be implemented.

No functional change is intended.

MFC after:	1 week
2014-12-21 22:18:30 +00:00
Jilles Tjoelker
88ef06f3a9 sh: Make sure output suitable as shell input is also printable.
Commands like 'export -p', 'set' and 'trap', and tracing enabled via 'set
-x' generate output suitable as shell input by adding quotes as necessary.

If there are control characters other than newline or invalid UTF-8
sequences, use $'...' and \OOO to display them safely.

The resulting output is not parsable by a strict POSIX.1-2008 shell but sh
from FreeBSD 9.0 and newer and many other shells can parse it.
2014-12-14 16:26:19 +00:00
Jilles Tjoelker
e753a1effb sh: Remove special case for '=' in set -x; always quote it in outqstr().
I plan to make set -x output always printable using $'...'; avoiding quoting
words containing '=' is not worth the extra complexity.
2014-11-30 20:12:47 +00:00
Baptiste Daroussin
12cd1730ee Convert bin/ to LIBADD, reduce overlinking allow to build all components as
static
2014-11-25 11:15:40 +00:00
Jilles Tjoelker
fafeab4393 sh: Prepend "$0: " to error messages if there is no command name. 2014-11-22 23:28:41 +00:00
Jilles Tjoelker
08dc8cf90c sh: Use DQSYNTAX only while expanding, not SQSYNTAX.
Quoting during expansion only cares about CCTL, which is the same for
DQSYNTAX and SQSYNTAX.
2014-11-22 16:03:18 +00:00
Simon J. Gerraty
9268022b74 Merge from head@274682 2014-11-19 01:07:58 +00:00
Jilles Tjoelker
b1f6059232 sh(1): Add/improve information about exit status of commands. 2014-11-14 17:11:28 +00:00
Jilles Tjoelker
945e15624b sh(1): Mention portability issue with shifting zero positional parameters.
Per Austin Group issue #459, shifting zero positional parameters may or may
not be considered an operand error (which causes the shell to exit in most
cases).
2014-11-07 21:30:16 +00:00
Jilles Tjoelker
5dff1efc27 sh: Fix corruption of CTL* bytes in positional parameters in redirection.
EXP_REDIR was not being checked for while expanding positional parameters in
redirection, so CTL* bytes were not being prefixed where they should be.

MFC after:	1 week
2014-10-31 22:28:10 +00:00
Jilles Tjoelker
3fb51b3a43 Treat IFS separators in "$*" as quoted.
This makes a difference if IFS starts with *, ?, [ or a CTL* byte.
2014-10-28 22:14:31 +00:00
Jilles Tjoelker
1bc2fdfabf sh: Make getopts memory-safe if with changing arguments.
POSIX does not permit to continuing a getopts loop with different
arguments. For parsing the positional parameters, we handle this case by
resetting the getopts state when the positional parameters are changed in
any way (and the getopts state is local to a function). However, in the
syntax getopts <optstring> <var> <arg...>, changes could lead to invalid
memory access.

In the syntax getopts <optstring> <var> <arg...>, store a copy of the
arguments and continue to use them until getopts is reset.
2014-10-26 17:50:33 +00:00
Jilles Tjoelker
0b4b9c8158 sh: Allow backslash-newline continuation in more places:
* directly after a $
 * directly after ${
 * between the characters of a multi-character operator token
 * within a parameter name
2014-10-19 11:59:15 +00:00
Jilles Tjoelker
334111e2df sh: Add some tests for backslash-newline continuation. 2014-10-17 21:52:57 +00:00
Jilles Tjoelker
3f9b4e9ab8 sh: Make parseredir() a proper function instead of an emulated nested
function.
2014-10-15 21:26:09 +00:00
Jilles Tjoelker
622fdf3236 sh: Remove more gotos. 2014-10-15 21:20:56 +00:00
Jilles Tjoelker
75e1716807 sh: Fix break/continue/return in multiline eval.
Example:
  eval $'return\necho bad'
2014-10-12 13:12:06 +00:00
Jilles Tjoelker
33c5acf038 sh: Eliminate some gotos. 2014-10-05 21:51:36 +00:00
Jilles Tjoelker
068dfa2d43 sh: Fix LINENO and prompt after $'\0 and newline. 2014-10-03 20:24:56 +00:00
Jilles Tjoelker
000dda7b47 sh(1): Clarify that assignments before commands do not affect expansions.
PR:		193759
MFC after:	1 week
2014-09-21 20:34:54 +00:00
Jilles Tjoelker
72238faa6a sh: Remove arbitrary length limit on << EOF markers.
This also simplifies the code.
2014-09-14 16:46:30 +00:00
Jilles Tjoelker
671a890e58 sh: Make checkend() a real function instead of an emulated nested function.
No functional change is intended, but the generated code is slightly
different.
2014-09-14 16:27:49 +00:00
Jilles Tjoelker
5545fadd84 sh: Add some const keywords. 2014-09-14 15:59:15 +00:00
Jilles Tjoelker
cd60e2c67d sh: Allow enabling job control without a tty in non-interactive mode.
If no tty is available, 'set -m' is still useful to put jobs in their own
process groups.
2014-09-04 21:48:33 +00:00
Jilles Tjoelker
dd6d480a3e sh: Remove two redundant (uintmax_t) casts.
Submitted by:	jmallett
2014-08-20 20:15:43 +00:00
Simon J. Gerraty
ee7b0571c2 Merge head from 7/28 2014-08-19 06:50:54 +00:00
Jilles Tjoelker
ef9e61785a sh: Avoid overflow in atoi() when parsing HISTSIZE.
Side effect: a non-numeric HISTSIZE now results in the default size (100)
instead of 0.
2014-08-17 19:36:56 +00:00
Jilles Tjoelker
d53f7f64f7 sh: Reject integer overflow in number and is_number. 2014-08-17 16:40:29 +00:00
Jilles Tjoelker
1c9c6ea481 sh: Don't hardcode relative paths in the tests stderr files.
These paths have had to be adjusted to changes in the testsuite runner
several times, so modify the tests to remove the need for such adjustment.

A cp in functional_test.sh is now unneeded, but this matters little in
performance.
2014-08-17 14:26:12 +00:00
Jilles Tjoelker
d5b14891eb sh: Mask off shift distance (<< and >>) in arithmetic.
In C, shift distances equal to or larger than the number of bits in the
operand result in undefined behaviour. As part of eliminating undefined
behaviour in arithmetic, mask off the distance like Java and JavaScript
specify and C on x86 usually does.

Assumption: conversion from unsigned to signed retains the two's complement
bits.
Assumption: uintmax_t has no padding bits.
2014-08-15 22:36:41 +00:00
Enji Cooper
12e2df3c36 Convert bin/sh/tests to ATF
The new code uses a "test discovery mechanism" to determine
what tests are available for execution

The test shell can be specified via:

  kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh

Sponsored by: EMC / Isilon Storage Division
Approved by: jmmv (mentor)
Reviewed by: jilles (maintainer)
2014-08-13 04:14:50 +00:00
Jilles Tjoelker
4d34663be3 sh: Allow arbitrarily large numbers in break and continue.
The argument is capped to loopnest, so strtol()'s [ERANGE] can be ignored.
2014-07-20 20:29:09 +00:00
Jilles Tjoelker
e61ae4ffc8 sh: Remove prefix() function. Use strncmp() instead. 2014-07-20 12:06:52 +00:00
Jilles Tjoelker
2ef146f14c sh: Deduplicate some code in ulimit builtin. 2014-07-19 14:06:23 +00:00
Brooks Davis
80189b3b09 Replace all uses of libncurses and libtermcap with their wide character
variants.  This allows usable file system images (i.e. those with both a
shell and an editor) to be created with only one copy of the curses library.

Exp-run:	antoine
PR:		189842
Discussed with:	bapt
Sponsored by:	DARPA, AFRL
2014-07-17 18:24:34 +00:00
Jilles Tjoelker
7b9104c0a9 sh: Correctly handle positional parameters beyond INT_MAX on 64-bit systems.
Currently, there can be no more than INT_MAX positional parameters. Make
sure to treat all higher ones as unset to avoid incorrect results and
crashes.

On 64-bit systems, our atoi() takes the low 32 bits of the strtol() and
sign-extends them.

On 32-bit systems, the call to atoi() returned INT_MAX for too high values
and there is not enough address space for so many positional parameters, so
there was no issue.
2014-07-12 21:54:11 +00:00
Jilles Tjoelker
5ddabb8348 sh: Consistently treat ${01} like $1.
Leading zeroes were ignored when checking whether a positional parameter is
set, but not when expanding its value. Ignore leading zeroes in any case.
2014-07-12 10:27:30 +00:00
Jilles Tjoelker
f1058ba09a sh: Add test for ${01} and ${010} that already works.
Although it is probably unwise to use this, POSIX is clear that leading
zeroes are permitted in positional parameters (and do not indicate octal).

Such positional parameters are checked for being unset and/or null
correctly, but their value is incorrectly expanded.
2014-07-08 22:04:44 +00:00
Jilles Tjoelker
725d072732 Don't install locale1.0 if MK_NLS == no.
The test locale1.0 depends on locale support; it is meaningless without a
working LC_MESSAGES.

I added an OptionalObsoleteFiles.inc entry.

PR:		181151
Submitted by:	Garrett Cooper (original version)
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-07-08 21:50:13 +00:00
Jilles Tjoelker
2d70c63720 sh: Fix overflow checking on 'ulimit' operand. 2014-07-05 21:50:59 +00:00
Jilles Tjoelker
b0762e4921 sh: Avoid undefined behaviour shifting negative values left in arithmetic.
With i386 base clang, arith_yacc.o remains unchanged.
2014-06-01 11:25:34 +00:00
Jilles Tjoelker
2ceda70233 sh: Simplify find_command() slightly. 2014-05-31 22:25:45 +00:00
Simon J. Gerraty
fae50821ae Updated dependencies 2014-05-16 14:09:51 +00:00
Jilles Tjoelker
c8fb3e69d0 sh: In getopts, unset OPTARG where POSIX says we should. 2014-05-10 19:18:49 +00:00
Jilles Tjoelker
8207fd5f81 sh: Add new tests to the Makefile. 2014-05-10 19:06:36 +00:00
Jilles Tjoelker
6e76445cf7 sh: Don't discard getopts state on unknown option or missing argument.
When getopts finds an invalid option or a missing option-argument, it should
not reset its state and should set OPTIND as normal. This is an old ash bug
that was fixed long ago in dash. Our behaviour now matches most other
shells.
2014-05-10 17:42:21 +00:00
Simon J. Gerraty
76b28ad6ab Updated dependencies 2014-05-10 05:16:28 +00:00
Jilles Tjoelker
8c4e5fc061 sh: Send getopts error messages to stderr, not stdout.
Adjust a testcase for this change.
2014-05-09 13:32:36 +00:00
Jilles Tjoelker
bc7f6652dd sh: Add more necessary INTOFF/INTON. 2014-05-09 13:27:30 +00:00
Simon J. Gerraty
cc3f4b9965 Merge from head 2014-05-08 23:54:15 +00:00
Jilles Tjoelker
5699ef2588 sh: Add some tests for normal use of getopts. 2014-05-07 21:45:25 +00:00
Warner Losh
c6063d0da8 Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.
2014-05-06 04:22:01 +00:00
Simon J. Gerraty
3b8f084595 Merge head 2014-04-28 07:50:45 +00:00
Jilles Tjoelker
d73dba75d6 sh: Fix memory leak with an assignment before a regular builtin.
MFC after:	1 week
2014-03-27 22:57:23 +00:00
Jilles Tjoelker
89d4f883a4 sh: Fix memory leak when trying to set a read only variable.
MFC after:	1 week
2014-03-27 22:52:26 +00:00
Jilles Tjoelker
1632bf1a88 sh: Fix possible memory leaks and double frees with unexpected SIGINT. 2014-03-26 20:43:40 +00:00
Jilles Tjoelker
728e552fdc sh: Don't overwrite old exit status if a PID is reused.
Only store exit status for a process if that process has not terminated yet.

Test (slow):
  exit 7 & p1=$!; until exit 8 & p2=$!; [ "$p1" = "$p2" ]; do wait "$p2";
  done; sleep 0.1; wait %1; echo $?
should write "7".
2014-03-20 22:38:13 +00:00
Jilles Tjoelker
7696168706 sh: Allow kill %job on jobs started without job control.
When killing a %job started without job control, kill all processes in it.
As with process groups and zombies, if any process in the job can be killed
or has already terminated, the command is successful.

This also fixes occasional failures of the builtins/kill1.0 test.
2014-03-15 14:58:48 +00:00
Jilles Tjoelker
61346cbdc7 sh: Add some consts. 2014-03-14 21:45:37 +00:00
Julio Merino
f5fd950e35 Make bsd.test.mk the only public mk fragment for the building of tests.
Change {atf,plain,tap}.test.mk to be internal implementation details of
bsd.test.mk.  Makefiles that build tests should now only include bsd.test.mk
and declaratively specify what they want to build, without worrying about
the internal implementation of the mk files.

The reason for this change is to permit building test programs of different
interfaces from a single directory, which is something I had a need for
while porting tests over from src/tools/regression/.

Additionally, this change makes it possible to perform some other requested
changes to bsd.test.mk in an easier manner.  Coming soon.
2014-03-14 08:56:19 +00:00
Jilles Tjoelker
4646e82dcd sh: Successfully do nothing when killing a terminated job.
If a job has terminated but is still known, silently do nothing when using
the kill builtin with the job specifier. Formerly, the shell called kill()
with the process group ID that might have been reused.
2014-03-08 19:44:34 +00:00
Jilles Tjoelker
a2cba42fc2 sh: Make argstr() return where it stopped and simplify expari() using this. 2014-03-04 22:30:38 +00:00
Jilles Tjoelker
ce16da82dd sh: Simplify expari().
Redo expari() like evalvar(). This makes the logic more understandable and
avoids possible problems if arithmetic expansion occurs if CTLESC characters
are not generated (looking backwards for CTLARI is not generally possible in
that case but the old code tried anyway).

This adds an extra argstr() recursion.
2014-03-02 22:59:34 +00:00
Jilles Tjoelker
5439648913 sh: Do not corrupt internal representation if LINENO inner expansion fails.
Example:
  f() { : ${LINENO+$((1/0))}; }
and call this function twice.
2014-02-27 16:54:43 +00:00
Jilles Tjoelker
85bf1d2f07 sh: Make expari() static. 2014-02-26 21:38:42 +00:00
Daichi GOTO
455b654622 sh: Add -h option to SYNOPSIS
Reviewed by:	jilles
MFC after:	soon
2014-02-25 03:05:43 +00:00
Jilles Tjoelker
48f49aac53 sh: Allow aliases to force alias substitution on the following word.
If an alias's value ends with a space or tab, the next word is also
checked for aliases.

This is a POSIX feature. It is useful with utilities like command and
nohup (alias them to themselves followed by a space).
2014-01-26 21:19:33 +00:00
Jilles Tjoelker
03e55809f4 sh: Add tests for alias names after another alias.
Since the first alias's value does not end with a blank, the next word
should not be checked for aliases.
2014-01-25 14:59:08 +00:00
Jilles Tjoelker
3d132810d2 sh: Do not depend on parse/execute split in new alias test. 2014-01-24 23:00:35 +00:00
Jilles Tjoelker
4b489a60e1 sh: Solve the alias recursion problem in a less hackish way.
Add the space to avoid alias recursion when the alias is expanded, not when
it is added.

As a result, displaying an alias via command -v, command -V or type no
longer erroneously appends a space. Adjust the tests so they now require
this bug to be absent.
2014-01-24 16:40:51 +00:00
Jilles Tjoelker
981a0f19d3 sh: Add test for nested alias. 2014-01-24 15:03:56 +00:00
Jilles Tjoelker
d78fdfde89 sh: Remove SIGWINCH handler and just check for resize before every read.
The SIGWINCH handler triggers breakage in libedit which is hard to fix; see
PR bin/169773.

Also, window size changes while a program is in foreground (and it rather
than sh will receive SIGWINCH) will now be picked up automatically.

Downside: it is now certain that a resize is only processed after pressing
<Enter>. If libedit is fixed, sh will most likely have to be changed also.

PR:		bin/180146
2014-01-14 22:56:25 +00:00
Julio Merino
9622238c09 Replace hand-crafted Kyuafiles with automatic generation.
Redo r260506 by using the new TEST_METADATA functionality of bsd.test.mk
to mark the sh(1) and test(1) tests as not supporting root.  This is to
get rid of hand-crafted Kyuafiles for these very simple cases.

MFC after:	5 days
2014-01-14 18:45:32 +00:00
Julio Merino
287083efce Run the sh(1) and test(1) tests as unprivileged.
One of the tests for test(1) fails and some of the tests for sh(1) are
silently bypassed when running as root.

To fix these tests and ensure they all run, mark the test programs for
sh(1) and test(1) as requiring an unprivileged user.  (This should and
will be the default in Kyua but isn't yet.)

MFC after:	1 week
2014-01-10 10:39:01 +00:00
Jilles Tjoelker
b14cfdf665 sh(1): Discourage use of -e.
Also, do not say that ! before a pipeline is an operator, because it is
syntactically a keyword.
2014-01-03 22:56:23 +00:00
Jilles Tjoelker
cac001aabe sh: Don't check input for non-whitespace if history is disabled.
preadbuffer() maintained a flag whether there was any non-whitespace
character. This flag is only useful when history is enabled (in that case,
lines containing only whitespace are not added to history). Instead, check
using strspn() when history is enabled.

There is an approximate 2% speedup when running
  sh -c '. /etc/rc.subr; . /etc/defaults/rc.conf; source_rc_confs'
with hot cache.
2013-12-27 15:52:18 +00:00
Jilles Tjoelker
8191e23727 sh: Simplify code related to PPID variable. 2013-12-25 16:14:02 +00:00
Jilles Tjoelker
7e81d0c8c3 sh: Initialize OPTIND=1 even if it came from the environment. 2013-12-24 22:38:24 +00:00
Jilles Tjoelker
bd28fb4d4c sh: Remove an unused variable. 2013-12-24 22:04:44 +00:00
Julio Merino
13de33a5dc Migrate tools/regression/bin/ tests to the new layout.
This change is a proof of concept on how to easily integrate existing
tests from the tools/regression/ hierarchy into the /usr/tests/ test
suite and on how to adapt them to the new layout for src.

To achieve these goals, this change:

- Moves tests from tools/regression/bin/<tool>/ to bin/<tool>/tests/.
- Renames the previous regress.sh files to legacy_test.sh.
- Adds Makefiles to build and install the tests and all their supporting
  data files into /usr/tests/bin/.
- Plugs the legacy_test test programs into the test suite using the new
  TAP backend for Kyua (appearing in 0.8) so that the code of the test
  programs does not have to change.
- Registers the new directories in the BSD.test.dist mtree file.

Reviewed by:	freebsd-testing
Approved by:	rpaulo (mentor)
2013-12-11 04:09:17 +00:00
Jilles Tjoelker
d1670d4228 sh: Split set -x output into a separate function. 2013-12-06 22:24:37 +00:00
Jilles Tjoelker
670dd3f08f sh: Prefer memcpy() to strcpy() in most cases. Remove the scopy macro. 2013-11-30 21:27:11 +00:00
Jilles Tjoelker
1b57cec7d9 sh: Make <&0 disable the </dev/null implicit in a background command.
Although <&0 does nothing, it is a redirection affecting standard input and
should therefore disable the </dev/null redirection implicit in a background
command.
2013-11-24 23:12:13 +00:00
Jilles Tjoelker
5d4d10e3e5 sh: Properly quote alias output from command -v.
An alias should be printed by command -v as a command line; therefore, make
the alias definition suitable for re-input to the shell.
2013-11-10 23:00:39 +00:00
Jilles Tjoelker
d2c068eaac sh(1),limits(1): Document kqueues (-k) rlimit. 2013-11-01 13:57:30 +00:00
Jilles Tjoelker
fd692a70aa sh: Reorder union node to reduce its size on 64-bit platforms. 2013-11-01 11:28:56 +00:00
Jilles Tjoelker
efd1946c35 sh: Allow trapping SIGINT/SIGQUIT after ignore because of '&'.
If job control is not enabled, background jobs started with  ... &  ignore
SIGINT and SIGQUIT so that they are not affected by such signals that are
intended for the foreground job. However, this should not prevent
reassigning a different action for these signals (as if the shell invocation
inherited these signal actions from its parent).

Austin group issue #751

Example:
  { trap - INT; exec sleep 10; } & wait
A Ctrl+C should terminate the sleep command.
2013-10-30 21:36:15 +00:00
Konstantin Belousov
85a0ddfd0b Add a resource limit for the total number of kqueues available to the
user.  Kqueue now saves the ucred of the allocating thread, to
correctly decrement the counter on close.

Under some specific and not real-world use scenario for kqueue, it is
possible for the kqueues to consume memory proportional to the square
of the number of the filedescriptors available to the process.  Limit
allows administrator to prevent the abuse.

This is kernel-mode side of the change, with the user-mode enabling
commit following.

Reported and tested by:	pho
Discussed with:	jmg
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2013-10-21 16:46:12 +00:00
Jilles Tjoelker
bab2982266 sh: Remove one syscall when waiting for a foreground job.
The getpgrp() call is unnecessary: if there is no job control then the
result was not used at all and if there is job control then we are not a
subshell and our process group ID is equal to our process ID (rootpid).
2013-10-18 12:35:12 +00:00
Simon J. Gerraty
34b33809b7 Updated dependencies 2013-10-13 00:24:00 +00:00
Simon J. Gerraty
d1d0158641 Merge from head 2013-09-05 20:18:59 +00:00
Jilles Tjoelker
2935c4cc5e sh: Make return return from the closest function or dot script.
Formerly, return always returned from a function if it was called from a
function, even if there was a closer dot script. This was for compatibility
with the Bourne shell which only allowed returning from functions.

Other modern shells and POSIX return from the function or the dot script,
whichever is closest.

Git 1.8.4's rebase --continue depends on the POSIX behaviour.

Reported by:	Christoph Mallon, avg
2013-09-04 22:10:16 +00:00
Jilles Tjoelker
b823fb59f1 sh: Fix race condition with signals and wait or set -T.
The change in r238888 was incomplete. It was still possible for a trapped
signal to arrive before the shell went to sleep (sigsuspend()) because a
check was missing or because the signal arrived before in_waitcmd was set.

On SMP, this bug sometimes caused the builtins/wait4.0 test to take 1 second
to execute; it then might or might not fail. On UP, the test almost always
failed.
2013-09-02 21:57:46 +00:00
Jilles Tjoelker
32187151d8 sh: Simplify list() in the parser.
The erflag argument was only used by old-style (``) command substitutions.
We can remove it and handle the special case in the command substitution
code.
2013-08-30 20:50:28 +00:00
Jilles Tjoelker
510739cc24 sh: Separate out nbinary allocation into a function. 2013-08-30 20:37:52 +00:00
Jilles Tjoelker
47752ed647 sh: Use makename() where possible. 2013-08-30 20:13:33 +00:00
Jilles Tjoelker
6ab99f87b7 sh: Add a function for the case where one token is required in the parse. 2013-08-30 13:25:15 +00:00
Jilles Tjoelker
65519ccb4d sh: Recognize "--" as end of options in type builtin.
This implementation makes minimal changes: command names starting with "-"
(other than "--") can still be queried normally.
2013-08-30 12:09:59 +00:00
Jilles Tjoelker
f52924b480 sh: Cast -1 to pointer rather than pointer to variable of wrong type.
NEOF needs to be a non-null pointer distinct from valid union node pointers.
It is not dereferenced.

The new NEOF is much like SIG_ERR except that it is an object pointer
instead of a function pointer.

The variable tokpushback can now be static.
2013-08-30 10:45:02 +00:00
Jilles Tjoelker
71828da5ff sh: Recognize "--" as end of options in alias builtin.
Aliases starting with "-" (which are non-POSIX) will need to be preceded by
an alias not starting with "-" or the newly added "--".
2013-08-25 11:42:53 +00:00
Jilles Tjoelker
3cfb11c41a sh: Disallow empty simple commands.
As per POSIX, a simple command must have at least one redirection,
assignment word or command word.

These occured in rare cases such as  eval "f()" .

The extension of allowing no commands inside { }, if, while, for, etc.
remains.
2013-08-25 10:57:48 +00:00
Jilles Tjoelker
7e6e930d1e sh: Reject ++ and -- in arithmetic.
POSIX does not require ++ and -- in arithmetic. It is probably more useful
to reject them than to treat ++x and --x as x silently.

Note that the behaviour of increment and decrement can be obtained via
(x+=1), ((x+=1)-1), (x-=1) and ((x-=1)+1).

PR:		bin/176444
2013-08-24 20:06:00 +00:00
Jilles Tjoelker
7cb5a0d863 sh: Do not prematurely discard stopped jobs in a wait builtin.
If a job is specified to 'wait', wait for it to complete. Formerly, in
interactive mode, the job was deleted if it stopped.

If no jobs are specified in interactive mode, 'wait' still waits for all jobs
to complete or stop.

In non-interactive mode, WUNTRACED is not passed to wait3() so stopped jobs
are not detected.

PR:		bin/181435
2013-08-24 09:57:32 +00:00
Jilles Tjoelker
9bb8ccd642 sh: Remove unnecessary reset functions.
These are already handled by exception handlers.
2013-08-16 20:24:41 +00:00
Jilles Tjoelker
f0ef49bbf4 sh: Recognize "--" as end of options in bg/fg/jobid builtins. 2013-08-16 13:56:43 +00:00
Jilles Tjoelker
056fd329b9 sh: Recognize "--" as end of options in local builtin. 2013-08-14 21:59:48 +00:00
Jilles Tjoelker
adc2e8dfb4 sh: Allow a lone redirection before '|', ';;' or ';&'.
Example: </dev/null | :

PR:		181240
MFC after:	1 week
2013-08-14 19:34:13 +00:00
Jilles Tjoelker
b6dda50adc sh: Remove an incorrect comment. 2013-07-25 20:50:35 +00:00
Jilles Tjoelker
0bdd387178 sh: Remove #define MKINIT.
MKINIT only served for the removed mkinit. Many variables can be static now.
2013-07-25 19:48:15 +00:00
Jilles Tjoelker
338b821b0f sh: Remove mkinit.
Replace the RESET blocks with regular functions and a reset() function that
calls them all.

This code generation tool is unusual and does not appear to provide much
benefit. I do not think isolating the knowledge about which modules need to
be reset is worth an almost 500-line build tool and wider scope for
variables used by the reset functions.

Also, relying on reset functions is often wrong: the cleanup should be done
in exception handlers so that no stale state remains after 'command eval'
and the like.
2013-07-25 15:08:41 +00:00
Jilles Tjoelker
f85336594c sh: Remove output.c's reset() handler.
These cleanup operations are not needed because they are already performed
after an optimized command substitution (whether there was an error or not).
2013-07-25 13:09:17 +00:00
Jilles Tjoelker
fd7d6d8a29 sh: Do not read from stdin if an error occurs during -i -c cmd.
Although using -i with -c does not seem very useful, it seems inappropriate
to read commands from the terminal in this case.

Side effect: if the -s -c extension is used and the -s option is turned off
using 'set +s' during the interactive part, the shell now exits after an
error or interrupt. Note that POSIX only specifies -s as option to sh, not
to set.

See also Austin Group issue #718.
2013-07-12 15:29:41 +00:00
Jilles Tjoelker
42580a3efa sh: Do not close(-1) if pipe() fails. 2013-06-28 21:47:08 +00:00
Jilles Tjoelker
0c35cec686 sh(1): A subshell environment has its own rlimits (ulimit).
This has always been the case and is intended (just like cd).

This matches Austin group issue #706.
2013-06-14 22:06:18 +00:00
Jilles Tjoelker
bd76c6b83f sh(1): Document new features in wait builtin.
PR:		176916
2013-06-05 19:54:28 +00:00
Jilles Tjoelker
79b1d31887 sh: Return status 127 for unknown jobs in wait builtin.
This is required by POSIX, at least for pids that are not known child
processes.

Other problems with job specifications still cause wait to abort with
exit status 2.

PR:		176916
2013-06-05 19:40:52 +00:00
Jilles Tjoelker
a4099656c3 sh: Allow multiple operands in wait builtin.
This is only part of the PR; the behaviour for unknown/invalid pids/jobs
remains unchanged (aborts the builtin with status 2).

PR:		176916
Submitted by:	Vadim Goncharov
2013-06-05 19:08:22 +00:00
Jilles Tjoelker
336e0c87fa sh: Remove linked list of stack marks.
The linked list of stack marks may cause problems if the allocation stack is
used between an exception and a higher-level popstackmark(), as it may then
touch a stack mark that is local to a function which has returned.

Also, the adjustment compares to a pointer passed to realloc(), which is
undefined behaviour.

Instead of adjusting stack marks when reallocating stack blocks, ensure that
such an adjustment is never necessary by fixing a small piece of memory in
place at a stack mark. This also simplifies the code.

To avoid the problems reported in bin/175922, it remains necessary to call
setstackmark() after popstackmark() if the stack mark remains in use.
2013-05-11 20:51:00 +00:00
Eitan Adler
6c7d832824 Fix two typos
Reviewed by:	jilles
2013-05-09 21:29:16 +00:00
Jilles Tjoelker
5aa6dfda1c sh: Use O_CLOEXEC and F_DUPFD_CLOEXEC instead of separate fcntl() call. 2013-05-05 10:51:40 +00:00
Jilles Tjoelker
c4539460e3 sh: Improve error handling in read builtin:
* If read -t times out, return status as if interrupted by SIGALRM
  (formerly 1).
* If a trapped signal interrupts read, return status 128+sig (formerly 1).
* If [EINTR] occurs but there is no trap, retry the read (for example
  because of a SIGWINCH in interactive mode).
* If a read error occurs, write an error message and return status 2.

As before, a variable assignment error returns 2 and discards the remaining
data read.
2013-05-03 15:28:31 +00:00
Jilles Tjoelker
58c73bab93 sh: Don't consider jobs -s/-p as reporting the status of jobs.
This ensures that something like j=$(jobs -p) does not prevent any
status from being written to the terminal.
2013-04-27 14:07:25 +00:00
Joel Dahl
1ca3beb91f Document a few expansions for the $PS1 and $PS2 environmental variables.
PR:		173410
Submitted by:	Derek Wood <ddwood@outlook.com>
Reviewed by:	jilles
2013-04-21 19:55:38 +00:00
Simon J. Gerraty
69e6d7b75e sync from head 2013-04-12 20:48:55 +00:00
Jilles Tjoelker
6e0f89a4b4 sh: Don't modify exit status when break/continue/return passes !.
This matches what would happen if  ! P  were to be replaced with
if P; then false; else true; fi.

Example:
  f() { ! return 0; }; f
2013-04-12 15:19:35 +00:00
Jilles Tjoelker
7d919c164d sh: Add const to nodesavestr(). 2013-04-07 16:28:36 +00:00
Jilles Tjoelker
c6a453a430 sh: Write as much into the heredoc pipe as possible, to avoid forking.
Use non-blocking I/O to write as much as the pipe will accept (often 64K,
but it can be as little as 4K), avoiding the need for the ugly PIPESIZE
constant. If PIPESIZE was set too high, a deadlock would occur.
2013-04-02 21:34:38 +00:00
Jilles Tjoelker
46c6b52dfb sh: Fix various compiler warnings.
It now passes WARNS=7 with clang on i386.

GCC 4.2.1 does not understand setjmp() properly so will always trigger
-Wuninitialized. I will not add the volatile keywords to suppress this.
2013-04-01 17:18:22 +00:00
Joel Dahl
5127efa399 Minor mdoc fix. 2013-03-29 08:12:09 +00:00
Jilles Tjoelker
54ccc8b588 sh(1): Mention possible ambiguities with $(( and ((.
In some other shells, things like $((a);(b)) are command substitutions.

Also, there are shells that have an extension ((ARITH)) that evaluates an
arithmetic expression and returns status 1 if the result is zero, 0
otherwise. This extension may lead to ambiguity with two subshells starting
in sequence.
2013-03-24 22:48:45 +00:00
Jilles Tjoelker
d70ad6f2d0 sh: Recognize "--" and explicitly reject options in wait builtin.
If syntactically invalid job identifiers are to be taken as jobs that exited
with status 127, this should not apply to options, so that we can add
options later if need be.
2013-03-15 20:29:31 +00:00
Simon J. Gerraty
7cf3a1c6b2 Updated dependencies 2013-03-11 17:21:52 +00:00
Jilles Tjoelker
3fe1119f98 sh: When executing a trap, keep exit status along with evalskip.
This ensures 'return' in a trap returns the correct status to the caller.

If evalskip is not set or if it is overridden by a previous evalskip, keep
the old behaviour of restoring the exit status from before the trap.
2013-03-03 17:33:59 +00:00
Jilles Tjoelker
25e0f0f577 sh: If a SIGINT or SIGQUIT interrupts "wait", return status 128+sig. 2013-02-23 22:50:57 +00:00
Jilles Tjoelker
e9e9223546 sh: Fix a crash with the stackmark code.
If a stack mark is set while the current stack block is empty, the stack
block may move later on (because of realloc()) and the stack mark needs to
be updated. This updating does not happen after popstackmark() has been
called; therefore, call setstackmark() again if the stack mark is still
being used.

For some reason, this only affects a few users. I cannot reproduce it. The
situation seems quite rare as well because an empty stack block would
usually be freed (by popstackmark()) before execution reaches a
setstackmark() call.

PR:		175922
Tested by:	KT Sin
2013-02-19 23:46:51 +00:00
Simon J. Gerraty
f5f7c05209 Updated dependencies 2013-02-16 01:23:54 +00:00
David E. O'Brien
d9a447559b Sync with HEAD. 2013-02-08 16:10:16 +00:00
Jilles Tjoelker
1767d529f2 sh: Simplify mksyntax and make it fit for cross-compiling.
Now it outputs fixed files, which use constants provided by the C standard
library to determine appropriate values for the target machine.

Before, mksyntax inspected the host machine which resulted in subtle
breakage if e.g. char is signed on the host and unsigned on the target such
as when cross-compiling on x86 for ARM.

Tested using -funsigned-char on amd64. Compiling build-tools without it and
sh itself with it causes various tests to fail without this change but not
with this change. With consistent -funsigned-char, tests pass with or
without this change.

The mksyntax program could be removed and syntax.c and syntax.h committed to
the repository.

Submitted by:	Christoph Mallon
MFC after:	2 weeks
2013-02-07 22:42:33 +00:00
Jilles Tjoelker
eb64a9137b sh: Fix a comment. 2013-02-07 21:24:10 +00:00
Xin LI
5be97e72e6 Catch TRACE parameters up with r238888. This change is only needed when
debugging is enabled.
2013-02-07 19:00:54 +00:00
Jilles Tjoelker
eaf7719957 sh: Do not test for digit_contig in mksyntax.
ISO/IEC 9899:1999 (E) 5.2.1p3 guarantees that the values of the characters
0123456789 are contiguous.

The generated syntax.c and syntax.h remain the same.

Submitted by:	Christoph Mallon
2013-02-05 22:54:09 +00:00
Jilles Tjoelker
4dc6bdd3e7 sh: Expand here documents in the current process.
Expand here documents at the same point other redirections are expanded but
use a non-fork subshell environment (like simple command substitutions) for
compatibility. Substitition errors result in an empty here document like
before.

As a result, a fork is avoided for short (<4K) expanded here documents.

Unexpanded here documents (with quoted end marker after <<) are not affected
by this change. They already only forked when >4K.

Side effects:
* Order of expansion is slightly different.
* Slow expansions are not executed in parallel with the redirected command.
* A non-fork subshell environment is subtly different from a forked process.
2013-02-03 15:54:57 +00:00
Jilles Tjoelker
58570ac4a5 sh: Prefer our character classification functions to <ctype.h>. 2013-01-31 22:10:57 +00:00
Jilles Tjoelker
f9b7cc523d sh: Show negated commands (!) in jobs output. 2013-01-31 21:50:44 +00:00
Joel Dahl
cb5a6a58cb Add FILES section.
Discussed with:	jilles
2013-01-22 18:02:58 +00:00
Joel Dahl
5dd78c9f36 Change the $ENV example to use .shrc instead of .shinit. This is consistent
with what we use in /usr/share/skel/dot.profile.

Discussed with: jilles
2013-01-20 22:25:25 +00:00
Jilles Tjoelker
84edde8b1a sh: Move some stackmarks to fix high memory usage in some loops.
If a loop contained certain commands (such as redirected compound commands),
the temporary memory for the redirection was not freed between iterations of
the loop but only after the loop.

Put a stackmark in evaltree(), freeing memory whenever a node has been
evaluated. Some other stackmarks are then redundant; remove them.

Example:
  while :; do { :; } </dev/null; done
2013-01-20 21:28:05 +00:00
Jilles Tjoelker
59e0cc8e4a sh: Remove mkinit's initialization routine.
Instead, call the only init function left directly from main().
2013-01-20 12:44:50 +00:00
Jilles Tjoelker
60a6bf2a5a sh: Replace an mkinit use with an initialization. 2013-01-19 22:12:08 +00:00
Jilles Tjoelker
523646ee5c sh: Simplify cd-hash interaction.
Instead of rechecking relative paths for all hashed utilities after a cd,
track if any utility in cmdtable depends on a relative path in PATH.
If there is such a utility, cd clears the entire table.

As a result, the '*' in hash no longer happens.
2013-01-14 16:40:50 +00:00
Jilles Tjoelker
925420d09c sh: Pass $? to command substitution containing compound/multiple commands.
Example:
  false; echo $(echo $?; :)
2013-01-14 12:20:55 +00:00
Jilles Tjoelker
31d396587e sh: Fix crash when parsing '{ } &'.
MFC after:	1 week
2013-01-13 19:26:33 +00:00
Jilles Tjoelker
2c090c7168 sh: Don't lose $? when backquoted command ends with semicolon or newline.
An empty simple command was added and overwrote the exit status with 0.

This affects `...` but not $(...).

Example:
  v=`false;`; echo $?
2013-01-13 19:19:40 +00:00
Jilles Tjoelker
cdbd40cbb2 sh: Prefer strsignal() to accessing sys_siglist directly.
Accessing sys_siglist directly requires rtld to copy it from libc to the sh
executable's BSS. Also, strsignal() will put in the signal number for
unknown signals (FreeBSD-specific) so we need not do that ourselves.

Unfortunately, there is no function for sys_signame.
2012-12-25 14:17:09 +00:00
Jilles Tjoelker
d6d66cfc00 sh: Detect and flag write errors on stdout in builtins.
If there is a write error on stdout, a message will be printed (to stderr)
and the exit status will be changed to 2 if it would have been 0 or 1.

PR:		bin/158206
2012-12-12 22:01:10 +00:00
Jilles Tjoelker
22bec20ba9 sh: Remove an unused variable. 2012-11-14 14:08:24 +00:00
Jilles Tjoelker
35dab8598a sh: Forward-declare struct alias instead of giving up type safety via void * 2012-11-11 15:13:24 +00:00
Jilles Tjoelker
e964872f18 sh: Fix two issues when an alias is redefined:
* The last character is not displayed.
 * If the alias ends with itself (as a word), an infinite memory-eating loop
   occurs.

If an alias is defined initially, a space is appended to avoid recursion but
this did not happen when an alias was later modified.

PR:		bin/173418
Submitted by:	Daniel F.
MFC after:	1 week
2012-11-08 13:33:48 +00:00
Jilles Tjoelker
d172408c7e sh: Change cmdtype in tblentry from short to signed char.
If this is a smaller type than int anyway, we can make it the smallest
possible.
2012-11-05 17:52:18 +00:00
Simon J. Gerraty
23090366f7 Sync from head 2012-11-04 02:52:03 +00:00
Jilles Tjoelker
422c281c83 sh: Use C99 flexible array instead of accessing array beyond bounds.
Although sufficient memory is available for a longer string in cmdname,
this is undefined behaviour anyway.

Side effect: for alignment reasons, an additional byte of memory is
allocated per hashed command.
2012-11-03 22:23:08 +00:00
Joel Dahl
a2442c2fa9 mdoc: don't nest displays. The markup here isn't adding anything anyway.
Fixes a mandoc lint warning.

Discussed with:	brueffer, Jason McIntyre <jmc@kerhand.co.uk>
2012-10-14 13:59:17 +00:00
Jilles Tjoelker
7cbda73825 sh: Prefer internal nextopt() to libc getopt().
This reduces code duplication and code size.

/usr/bin/printf is not affected.

Side effect: different error messages when certain builtins are passed
invalid options.
2012-09-15 21:56:30 +00:00
Jilles Tjoelker
d902a66b99 sh: Reduce code duplication: use setinputfile instead of open/setinputfd
combination.

MFC after:	2 weeks
2012-09-14 15:22:15 +00:00
Joel Dahl
afc5a69a1d Minor mdoc fix. 2012-09-11 17:57:03 +00:00
Jilles Tjoelker
36f7fbeae8 sh: Remove XXX comment about removing nextopt().
Using nextopt() avoids depending on the BSD-specific optreset feature in
getopt() and reduces code size (both source and binary).
2012-09-08 19:24:03 +00:00
Marcel Moolenaar
7750ad47a9 Sync FreeBSD's bmake branch with Juniper's internal bmake branch.
Requested by: Simon Gerraty <sjg@juniper.net>
2012-08-22 19:25:57 +00:00
Jilles Tjoelker
1794251add sh: Fix EINTR race condition in "wait" and "set -T" using sigsuspend().
When waiting for child processes using "wait" or if "set -T" is in effect, a
signal interrupts the wait. Make sure there is no window where the signal
handler may be invoked (setting a flag) just before going to sleep.

There is a similar race condition in the shell language, but scripts can
avoid it by exiting from the trap handler or enforcing synchronization using
a fifo.

If SIGCHLD is not trapped, a signal handler must be installed for it. Only
install this handler for the duration of the wait to avoid triggering
unexpected [EINTR] errors elsewhere.

Note that for some reason only SIGINT and SIGQUIT interrupt a "wait"
command. This remains the case.
2012-07-29 18:04:38 +00:00
Jilles Tjoelker
fb05913c5d sh: Do not ask for stopped/continued processes if we do not need them
rather than retrying wait3 if they happen.
2012-07-28 15:13:48 +00:00
Jilles Tjoelker
61fb716ac9 sh: Inline waitproc() into its only caller. 2012-07-28 14:56:50 +00:00
Jilles Tjoelker
faa787d323 sh: Track continued jobs (even if not continued by bg or fg).
This uses wait3's WCONTINUED flag.

There is no message for this. The change is visible in "jobs" or if the job
stops again.
2012-07-28 14:32:55 +00:00
Jilles Tjoelker
fa7ccda2c4 sh: Reset pendingsigs before checking pending traps, not after.
Otherwise, a signal arriving at exactly the right moment might not be
processed until another signal arrived.
2012-07-15 11:18:52 +00:00
Jilles Tjoelker
184dcee217 sh: Remove unused variable in_dowait. 2012-07-15 10:49:16 +00:00
Jilles Tjoelker
84fbdd8ca0 sh: Expand assignment-like words specially for export/readonly/local.
Examples:
  export x=~
now expands the tilde
  local y=$1
is now safe, even if $1 contains IFS characters or metacharacters.

For a word to "look like an assignment", it must start with a name followed
by an equals sign, none of which may be quoted.

The special treatment applies when the first word (potentially after
"command") is "export", "readonly" or "local". There may be quoting
characters but no expansions. If "local" is overridden with a function there
is no special treatment ("export" and "readonly" cannot be overridden with a
function).

If things like
  local arr=(1 2 3)
are ever allowed in the future, they cannot call a "local" function. This
would either be a run-time error or it would call the builtin.

This matches Austin Group bug #351, planned for the next issue of POSIX.1.

PR:		bin/166771
2012-07-15 10:19:43 +00:00
Pedro F. Giffuni
00c43e0ca4 Merge libedit adjustment from NetBSD.
On recent versions of NetBSD's libedit, el_gets
now sets el_len to -1 on error so we can
distinguish between a NULL string and an error.

This fixes sh from exiting with newer versions
of libedit now allowing EINTR to return.

Obtained from:	NetBSD
Reviewed by:	jilles
MFC after:	3 weeks
2012-07-11 22:17:58 +00:00
Marcel Moolenaar
937b0a997b Work better with how make/bmake works:
1.  Avoid a cd back into ${.CURDIR} to run mkbuiltins when we know make
    will first cd into ${.OBJDIR}. Keep the cwd to what make sets it to.
2.  Don't tell mkbuiltins where to write to (= ${.OBJDIR}), but where to
    get sources from (= ${.CURDIR}). This to compensate for point 1.

This fixes a problem with bmake's mk files that optimize ${.OBJDIR} to
expand to "." after changing cwd, not taking into account that the
target is pretty much undoing that and not getting the full path to the
object tree anymore.
2012-05-24 19:48:15 +00:00
Jilles Tjoelker
ad859ed8cf sh: Remove an unused header.
The read builtin no longer does things with termios.
2012-05-15 22:50:47 +00:00
Joel Dahl
35471bf8ef Minor mdoc nits. 2012-05-13 14:16:04 +00:00
Joel Dahl
748611c9c0 mdoc: fix column names, indentation, column separation within each row, and
quotation. Also make sure we have the same amount of columns in each row as
the number of columns we specify in the head arguments.

Reviewed by:	brueffer
2012-04-07 09:05:30 +00:00
Jilles Tjoelker
b95fca474d sh: Fix build with -DDEBUG=2.
Reported by:	Kristof Provost
MFC after:	1 week
2012-04-02 17:16:24 +00:00
Eitan Adler
50d675f7a9 Remove trailing whitespace per mdoc lint warning
Disussed with:	gavin
No objection from:	doc
Approved by:	joel
MFC after:	3 days
2012-03-29 05:02:12 +00:00
Joel Dahl
904d0726c8 Remove superfluous paragraph macro. 2012-03-25 09:20:14 +00:00
Jilles Tjoelker
c0b3cf0676 sh: Make 'hash' return 1 if at least one utility is not found.
Reported by:	lme
2012-02-11 21:06:45 +00:00
Jilles Tjoelker
33233ec76e sh: Fix swapped INTON/INTOFF.
A possible consequence of this bug was a memory leak if SIGINT arrived
during a 'set' command (listing variables).

MFC after:	1 week
2012-02-04 23:29:07 +00:00
Jilles Tjoelker
caf29fab8c sh: Use vfork in a few common cases.
This uses vfork() for simple commands and command substitutions containing a
single simple command, invoking an external program under certain conditions
(no redirections or variable assignments, non-interactive shell, no job
control). These restrictions limit the amount of code executed in a vforked
child.

There is a large speedup (for example 35%) in microbenchmarks. The
difference in buildkernel is smaller (for example 0.5%) but still
statistically significant. See
http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/037581.html
for some numbers.

The use of vfork() can be disabled by setting a variable named
SH_DISABLE_VFORK.
2012-02-04 23:12:14 +00:00
Philippe Charnier
2fae4c3d95 Add prototypes, ANSIfy functions definitions to reduce WARNS=6 output. 2012-01-25 08:42:19 +00:00
Jilles Tjoelker
6eff4a75d5 sh: Fix $? in the first command of a 'for'.
In the first command of a 'for', $? should be the exit status of the last
pipeline (command substitution in the word list or command before 'for'),
not always 0.
2012-01-22 14:00:33 +00:00
Jilles Tjoelker
0105d7224d sh: Remove "kill" example function, which is superseded by the kill builtin
MFC after:	1 week
2012-01-21 20:39:17 +00:00
Jean-Sébastien Pédron
01a43bcf7b sh: Fix execution of multiple statements in a trap when evalskip is set
Before this fix, only the first statement of the trap was executed if
evalskip was set. This is for example the case when:
    o  "-e" is set for this shell
    o  a trap is set on EXIT
    o  a function returns 1 and causes the script to abort

Reviewed by:	jilles
MFC after:	2 weeks
2012-01-16 11:07:46 +00:00
Jilles Tjoelker
93fcb2511b sh: Fix some bugs with exit status from case containing ;&.
Also, rework evalcase() to not evaluate any tree. Instead, return the
NCLISTFALLTHRU node and handle it in evaltree().

Fixed bugs:

* If a ;& list with non-zero exit status is followed by an empty ;; or final
  list, the exit status of the case command should be equal to the exit
  status of the ;& list, not 0.

* An empty ;& case should not reset $?.
2012-01-15 21:39:38 +00:00