Examples:
export x=~
now expands the tilde
local y=$1
is now safe, even if $1 contains IFS characters or metacharacters.
For a word to "look like an assignment", it must start with a name followed
by an equals sign, none of which may be quoted.
The special treatment applies when the first word (potentially after
"command") is "export", "readonly" or "local". There may be quoting
characters but no expansions. If "local" is overridden with a function there
is no special treatment ("export" and "readonly" cannot be overridden with a
function).
If things like
local arr=(1 2 3)
are ever allowed in the future, they cannot call a "local" function. This
would either be a run-time error or it would call the builtin.
This matches Austin Group bug #351, planned for the next issue of POSIX.1.
PR: bin/166771
On recent versions of NetBSD's libedit, el_gets
now sets el_len to -1 on error so we can
distinguish between a NULL string and an error.
This fixes sh from exiting with newer versions
of libedit now allowing EINTR to return.
Obtained from: NetBSD
Reviewed by: jilles
MFC after: 3 weeks
1. Avoid a cd back into ${.CURDIR} to run mkbuiltins when we know make
will first cd into ${.OBJDIR}. Keep the cwd to what make sets it to.
2. Don't tell mkbuiltins where to write to (= ${.OBJDIR}), but where to
get sources from (= ${.CURDIR}). This to compensate for point 1.
This fixes a problem with bmake's mk files that optimize ${.OBJDIR} to
expand to "." after changing cwd, not taking into account that the
target is pretty much undoing that and not getting the full path to the
object tree anymore.
quotation. Also make sure we have the same amount of columns in each row as
the number of columns we specify in the head arguments.
Reviewed by: brueffer
This uses vfork() for simple commands and command substitutions containing a
single simple command, invoking an external program under certain conditions
(no redirections or variable assignments, non-interactive shell, no job
control). These restrictions limit the amount of code executed in a vforked
child.
There is a large speedup (for example 35%) in microbenchmarks. The
difference in buildkernel is smaller (for example 0.5%) but still
statistically significant. See
http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/037581.html
for some numbers.
The use of vfork() can be disabled by setting a variable named
SH_DISABLE_VFORK.
In the first command of a 'for', $? should be the exit status of the last
pipeline (command substitution in the word list or command before 'for'),
not always 0.
Before this fix, only the first statement of the trap was executed if
evalskip was set. This is for example the case when:
o "-e" is set for this shell
o a trap is set on EXIT
o a function returns 1 and causes the script to abort
Reviewed by: jilles
MFC after: 2 weeks
Also, rework evalcase() to not evaluate any tree. Instead, return the
NCLISTFALLTHRU node and handle it in evaltree().
Fixed bugs:
* If a ;& list with non-zero exit status is followed by an empty ;; or final
list, the exit status of the case command should be equal to the exit
status of the ;& list, not 0.
* An empty ;& case should not reset $?.
* If no pattern is matched, POSIX says the exit status shall be 0 (even if
there are command substitutions).
* If a pattern is matched and there are no command substitutions, the first
command should see the $? from before the case command, not always 0.
The errno message display added in r222292 did not take attempting to
cd to a non-directory or something that cannot be stat()ed into account.
PR: bin/164070
MFC after: 10 days
This improves performance for globs where a slash or another component
follows a component with metacharacters by eliminating unnecessary attempts
to open directories that are not.
This is not necessary: errors are already caught in evalbackcmd() and
forcelocal handles changes to variables.
Note that this depends on r223024.
MFC after: 4 weeks
Free expanded case text before executing commands.
Remove impossible evalskip checks (expanding an argument cannot set
evalskip anymore since $(break) and the like are properly executed in a
subshell environment).
Because sh executes commands in subshell environments without forking in
more and more cases (particularly from 8.0 on), it makes sense to describe
subshell environments more precisely using ideas from POSIX, together with
some FreeBSD-specific items.
In particular, the hash and times builtins may not behave as if their state
is copied for a subshell environment while leaving the parent shell
environment unchanged.
* Shell patterns are also for ${var#pat} and the like.
* An '!' by itself will not trigger pathname generation so do not call it a
meta-character, even though it has a special meaning directly after an
'['.
* Character ranges are locale-dependent.
* A '^' will complement a character class like '!' but is non-standard.
MFC after: 1 week
POSIX requires a -h option to sh and set, to locate and remember utilities
invoked by functions as they are defined. Given that this
locate-and-remember process is optional elsewhere, it seems safe enough to
make this option do nothing.
POSIX does not specify a long name for this option. Follow ksh in calling it
"trackall".
Replacing ;; with the new control operator ;& will cause the next list to be
executed as well without checking its pattern, continuing until a list ends
with ;; or until the end of the case statement. This is like omitting
"break" in a C "switch" statement.
The sequence ;& was formerly invalid.
This feature is proposed for the next POSIX issue in Austin Group issue
#449.
The eval special builtin now runs the code with EV_EXIT if it was run
with EV_EXIT itself.
In particular, this eliminates one fork when a command substitution contains
an eval command that ends with an external program or a subshell.
This is similar to what r220978 did for functions.
Have mkbuiltins write the prototypes for the *cmd functions to builtins.h
instead of builtins.c and include builtins.h in more .c files instead of
duplicating prototypes for *cmd functions in other headers.
In optimized command substitution, save and restore any variables changed by
expansions (${var=value} and $((var=assigned))), instead of trying to
determine if an expansion may cause such changes.
If $! is referenced in optimized command substitution, do not cause jobs to
be remembered longer.
This fixes $(jobs $!) again, simplifies the man page and shortens the code.
When I added UTF-8 support in r221646, the LC_COLLATE-based ordering broke
because of sign extension of char.
Because of libc restrictions, this does not work for UTF-8. For UTF-8
locales, ranges always use character code order.
In most cases, login shells are started from the home directory, but not in
all, such as xterm -ls.
This commit depends on r222957 for read_profile() performing parameter
expansion.
PR: bin/50569
The function name expandstr() and the general idea of doing this kind of
expansion by treating the text as a here document without end marker is from
dash.
All variants of parameter expansion and arithmetic expansion also work (the
latter is not required by POSIX but it does not take extra code and many
other shells also allow it).
Command substitution is prevented because I think it causes too much code to
be re-entered (for example creating an unbounded recursion of trace lines).
Unfortunately, our LINENO is somewhat crude, otherwise PS4='$LINENO+ ' would
be quite useful.
The "exp" builtin is undocumented, non-standard and not very useful.
If exp's return value is not used, something like
VAR=$(exp EXPRESSION)
is equivalent to
VAR=$((EXPRESSION))
except that errors in the expression are fatal and quoting special
characters is not needed in the latter case.
If exp's return value is used, something like
if exp EXPRESSION >/dev/null
can be replaced by
if [ $((EXPRESSION)) -ne 0 ]
with similar differences.
The exp-run showed that "let" is close enough to bash's and ksh's builtin
that removing it would break a few ports. Therefore, "let" remains in 9.x.
PR: bin/104432
Exp-run done by: pav (with some other sh(1) changes)
CDPATH should be ignored not only for pathnames starting with '/' but also
for pathnames whose first component is '.' or '..'.
The man page already describes this behaviour.
If IFS is null, unquoted $@/$* should still expand to separate words.
This differs from quoted $@ (which does not depend on IFS) in that pathname
generation is performed and empty words are removed.
If the length of a directory in PATH together with the given filename
exceeded FILENAME_MAX (which may happen even for pathnames that work), a
static buffer was overflown.
The static buffer is unnecessary, we can use the stalloc() stack.
Obtained from: NetBSD
MFC after: 1 week
This reflects failure to determine the pathname of the new directory in the
exit status (1). Normally, cd returns successfully if it did chdir() and the
call was successful.
In POSIX, -e only has meaning with -P; because our -L is not entirely
compliant and may fall back to -P mode, -e has some effect with -L as well.
This is sometimes used with eval or old-style command substitution, and most
shells other than ash derivatives allow it.
It can also be used with scripts that violate POSIX's requirement on the
application that they end in a newline (scripts must be text files except
that line length is unlimited).
Example:
v=`cat <<EOF
foo
EOF`
echo $v
This commit does not add support for the similar construct with new-style
command substitution, like
v=$(cat <<EOF
foo
EOF)
This continues to require a newline after the terminator.
Because we have no iconv in base, support for other charsets is not
possible.
Note that \u/\U are processed using the locale that was active when the
shell started. This is necessary to avoid behaviour that depends on the
parse/execute split (for example when placing braces around an entire
script). Therefore, UTF-8 encoding is implemented manually.
?, [...] patterns match codepoints instead of bytes. They do not match
invalid sequences. [...] patterns must not contain invalid sequences
otherwise they will not match anything. This is so that ${var#?} removes the
first codepoint, not the first byte, without putting UTF-8 knowledge into
the ${var#pattern} code. However, * continues to match any string and an
invalid sequence matches an identical invalid sequence. (This differs from
fnmatch(3).)
This ensures that mbrtowc(3) can be used directly once it has been verified
that there is no CTL* byte. Dealing with a CTLESC byte within a multibyte
character would be complicated.
The new values do occur in iso-8859-* encodings. This decreases efficiency
slightly but should not affect correctness.
Caveat: Updating across this change and rebuilding without cleaning may
yield a subtly broken sh binary. By default, make buildworld will clean and
avoid problems.
A string between $' and ' may contain backslash escape sequences similar to
the ones in a C string constant (except that a single-quote must be escaped
and a double-quote need not be). Details are in the sh(1) man page.
This construct is useful to include unprintable characters, tabs and
newlines in strings; while this can be done with a command substitution
containing a printf command, that needs ugly workarounds if the result is to
end with a newline as command substitution removes all trailing newlines.
The construct may also be useful in future to describe unprintable
characters without needing to write those characters themselves in 'set -x',
'export -p' and the like.
The implementation attempts to comply to the proposal for the next issue of
the POSIX specification. Because this construct is not in POSIX.1-2008,
using it in scripts intended to be portable is unwise.
Matching the minimal locale support in the rest of sh, the \u and \U
sequences are currently not useful.
Exp-run done by: pav (with some other sh(1) changes)
Note that this only applies to variables that are actually used.
Things like (0 && unsetvar) do not cause an error.
Exp-run done by: pav (with some other sh(1) changes)
In particular, this makes things like ${#foo[0]} and ${#foo[@]} errors
rather than silent equivalents of ${#foo}.
PR: bin/151720
Submitted by: Mark Johnston
Exp-run done by: pav (with some other sh(1) changes)
For backgrounded pipelines and subshells, the previous value of $? was being
preserved, which is incorrect.
For backgrounded simple commands containing a command substitution, the
status of the last command substitution was returned instead of 0.
If fork() fails, this is an error.
If the -p option is turned off, privileges from a setuid or setgid binary
are dropped. Make sure to check if this succeeds. If it fails, this is an
error which will cause the shell to abort except in interactive mode or if
'command' was used to make 'set' or an outer 'eval' or '.' non-special.
Note that taking advantage of this feature and writing setuid shell scripts
seems unwise.
MFC after: 1 week
If EV_EXIT causes an exit, use the exception mechanism to unwind
redirections and local variables. This way, if the final command is a
redirected command, an EXIT trap now executes without the redirections.
Because of these changes, EV_EXIT can now be inherited by the body of a
function, so do so. This means that a function no longer prevents a fork
before an exec being skipped, such as in
f() { head -1 /etc/passwd; }; echo $(f)
Wrapping a single builtin in a function may still cause an otherwise
unnecessary fork with command substitution, however.
An exit command or -e failure still invokes the EXIT trap with the
original redirections and local variables in place.
Note: this depends on SHELLPROC being gone. A SHELLPROC depended on
keeping the redirections and local variables and only cleaning up the
state to restore them.
This is only a problem if IFS contains digits, which is unusual but valid.
Because of an incorrect fix for PR bin/12137, "${#parameter}" was treated
as ${#parameter}. The underlying problem was that "${#parameter}"
erroneously added CTLESC bytes before determining the length. This
was properly fixed for PR bin/56147 but the incorrect fix was not backed
out.
Reported by: Seeker on forums.freebsd.org
MFC after: 2 weeks
POSIX does not require the shell to fork for a subshell environment, and we
use that possibility in various ways (command substitutions with a single
command and most subshells that are the final command of a shell process).
Therefore do not tie subshells to forking in the man page.
Command substitutions with expansions are a bit strange, causing a fork for
$(...$(($x))...) because $x might expand to y=2; they will probably be
changed later but this is how they work now.
These already worked: $# ${#} ${##} ${#-} ${#?}
These now work as well: ${#+word} ${#-word} ${##word} ${#%word}
There is an ambiguity in the standard with ${#?}: it could be the length of
$? or it could be $# giving an error in the (impossible) case that it is not
set. We continue to use the former interpretation as it seems more useful.
New features:
* proper lazy evaluation of || and &&
* ?: ternary operator
* executable is considerably smaller (8K on i386) because lex and yacc are
no longer used
Differences from dash:
* arith_t instead of intmax_t
* imaxdiv() not used
* unset or null variables default to 0
* let/exp builtin (undocumented, will probably be removed later)
Obtained from: dash
* In {(...) <redir1;} <redir2, do not drop redir1.
* Maintain the difference between (...) <redir and {(...)} <redir:
In (...) <redir, the redirection is performed in the child, while in
{(...)} <redir it should be performed in the parent (like {(...); :;}
<redir)
POSIX requires this and it is simpler than the previous code that remembered
command locations when appending directories to PATH.
In particular,
PATH=$PATH
is no longer a no-op but discards all cached command locations.
If execve() returns an [ENOEXEC] error, check if the file is binary before
trying to execute it using sh. A file is considered binary if at least one
of the first 256 bytes is '\0'.
In particular, trying to execute ELF binaries for the wrong architecture now
fails with an "Exec format error" message instead of syntax errors and
potentially strange results.
These are called "shell procedures" in the source.
If execve() failed with [ENOEXEC], the shell would reinitialize itself
and execute the program as a script. This requires a fair amount of code
which is not frequently used (most scripts have a #! magic number).
Therefore just execute a new instance of sh (_PATH_BSHELL) to run the
script.
