Commit Graph

2740 Commits

Author SHA1 Message Date
Jilles Tjoelker
50df342447 sh(1): Update BUGS section for UTF-8 support. 2011-05-08 14:03:44 +00:00
Jilles Tjoelker
7cc6b3df80 sh: Add UTF-8 support to pattern matching.
?, [...] patterns match codepoints instead of bytes. They do not match
invalid sequences. [...] patterns must not contain invalid sequences
otherwise they will not match anything. This is so that ${var#?} removes the
first codepoint, not the first byte, without putting UTF-8 knowledge into
the ${var#pattern} code. However, * continues to match any string and an
invalid sequence matches an identical invalid sequence. (This differs from
fnmatch(3).)
2011-05-08 11:32:20 +00:00
Jilles Tjoelker
4c244ed255 sh: Add UTF-8 support to ${#var}.
If the current locale uses UTF-8, ${#var} counts codepoints (more precisely,
bytes b with (b & 0xc0) != 0x80).
2011-05-07 14:32:16 +00:00
Jilles Tjoelker
6ed74a0a1c sh: Track if the current locale's charset is UTF-8 or not. 2011-05-06 22:31:27 +00:00
Jilles Tjoelker
c76aed958b sh: Change the CTL* bytes to ones invalid in UTF-8.
This ensures that mbrtowc(3) can be used directly once it has been verified
that there is no CTL* byte. Dealing with a CTLESC byte within a multibyte
character would be complicated.

The new values do occur in iso-8859-* encodings. This decreases efficiency
slightly but should not affect correctness.

Caveat: Updating across this change and rebuilding without cleaning may
yield a subtly broken sh binary. By default, make buildworld will clean and
avoid problems.
2011-05-06 20:45:50 +00:00
Jilles Tjoelker
a62ab0274a sh: Add $'quoting' (C-style escape sequences).
A string between $' and ' may contain backslash escape sequences similar to
the ones in a C string constant (except that a single-quote must be escaped
and a double-quote need not be). Details are in the sh(1) man page.

This construct is useful to include unprintable characters, tabs and
newlines in strings; while this can be done with a command substitution
containing a printf command, that needs ugly workarounds if the result is to
end with a newline as command substitution removes all trailing newlines.

The construct may also be useful in future to describe unprintable
characters without needing to write those characters themselves in 'set -x',
'export -p' and the like.

The implementation attempts to comply to the proposal for the next issue of
the POSIX specification. Because this construct is not in POSIX.1-2008,
using it in scripts intended to be portable is unwise.

Matching the minimal locale support in the rest of sh, the \u and \U
sequences are currently not useful.

Exp-run done by: pav (with some other sh(1) changes)
2011-05-05 20:55:55 +00:00
Jilles Tjoelker
3937fc9c26 sh: Apply set -u to variables in arithmetic.
Note that this only applies to variables that are actually used.
Things like (0 && unsetvar) do not cause an error.

Exp-run done by: pav (with some other sh(1) changes)
2011-05-04 22:12:22 +00:00
Jilles Tjoelker
fc0818fef3 sh: Detect an error for ${#var<GARBAGE>}.
In particular, this makes things like ${#foo[0]} and ${#foo[@]} errors
rather than silent equivalents of ${#foo}.

PR:		bin/151720
Submitted by:	Mark Johnston
Exp-run done by: pav (with some other sh(1) changes)
2011-05-04 21:49:34 +00:00
Ruslan Ermilov
a35a58824b Don't call -f option's argument "stdin".
MFC after:	3 days
2011-05-03 10:08:11 +00:00
Jilles Tjoelker
03b3a844d0 sh: Set $? to 0 for background commands.
For backgrounded pipelines and subshells, the previous value of $? was being
preserved, which is incorrect.

For backgrounded simple commands containing a command substitution, the
status of the last command substitution was returned instead of 0.

If fork() fails, this is an error.
2011-04-25 20:54:12 +00:00
Jilles Tjoelker
92a1de471c sh: Check setuid()/setgid() return values.
If the -p option is turned off, privileges from a setuid or setgid binary
are dropped. Make sure to check if this succeeds. If it fails, this is an
error which will cause the shell to abort except in interactive mode or if
'command' was used to make 'set' or an outer 'eval' or '.' non-special.

Note that taking advantage of this feature and writing setuid shell scripts
seems unwise.

MFC after:	1 week
2011-04-25 10:14:29 +00:00
Jilles Tjoelker
b7b23db5e2 sh: Remove duplicate code resetting uid/gid for set +p/+o privileged.
MFC after:	1 week
2011-04-25 10:08:34 +00:00
Jilles Tjoelker
45496405c6 sh: Allow EV_EXIT through function calls, make {...} <redir more consistent.
If EV_EXIT causes an exit, use the exception mechanism to unwind
redirections and local variables. This way, if the final command is a
redirected command, an EXIT trap now executes without the redirections.

Because of these changes, EV_EXIT can now be inherited by the body of a
function, so do so. This means that a function no longer prevents a fork
before an exec being skipped, such as in
  f() { head -1 /etc/passwd; }; echo $(f)

Wrapping a single builtin in a function may still cause an otherwise
unnecessary fork with command substitution, however.

An exit command or -e failure still invokes the EXIT trap with the
original redirections and local variables in place.

Note: this depends on SHELLPROC being gone. A SHELLPROC depended on
keeping the redirections and local variables and only cleaning up the
state to restore them.
2011-04-23 22:28:56 +00:00
Jilles Tjoelker
caa7ccdc54 sh: Do not word split "${#parameter}".
This is only a problem if IFS contains digits, which is unusual but valid.

Because of an incorrect fix for PR bin/12137, "${#parameter}" was treated
as ${#parameter}. The underlying problem was that "${#parameter}"
erroneously added CTLESC bytes before determining the length. This
was properly fixed for PR bin/56147 but the incorrect fix was not backed
out.

Reported by:	Seeker on forums.freebsd.org
MFC after:	2 weeks
2011-04-20 22:24:54 +00:00
Edward Tomasz Napierala
b9ff20ab7e Document problems with -d/-w and the fact that -X is the default.
Suggested by:	arundel@
Reviewed by:	arundel@
2011-04-18 19:20:47 +00:00
Edward Tomasz Napierala
649fde88ff Get rid of DSIZ; instead just call the sizing function if provided. 2011-04-12 20:10:15 +00:00
Edward Tomasz Napierala
181b4eebb9 Make it possible to use permission sets (full_set, modify_set, read_set
and write_set) with setfacl(1).

PR:		kern/154113
Submitted by:	Shawn Webb <lattera at gmail dot com> (earlier version)
MFC after:	1 month
2011-04-09 07:42:25 +00:00
Edward Tomasz Napierala
f9db2550b2 Add proper width calculation for time fields (time, cputime and usertime).
This fixes the ugly overflow in "ps aux" output for "[idle]".
2011-03-24 20:15:42 +00:00
Edward Tomasz Napierala
3bf92decd3 Make "LOGIN" and "CLASS" columns width scale properly instead of wasting space. 2011-03-24 17:20:24 +00:00
Jilles Tjoelker
ef89d04f13 sh(1): Describe subshell environment, command substitution more correctly.
POSIX does not require the shell to fork for a subshell environment, and we
use that possibility in various ways (command substitutions with a single
command and most subshells that are the final command of a shell process).
Therefore do not tie subshells to forking in the man page.

Command substitutions with expansions are a bit strange, causing a fork for
$(...$(($x))...) because $x might expand to y=2; they will probably be
changed later but this is how they work now.
2011-03-20 23:52:45 +00:00
Konstantin Belousov
a870bf2c31 Implement the usertime and systime keywords for ps, printing the
corresponding times reported by getrusage().

Submitted by:	Dan Nelson <dnelson allantgroup com>
MFC after:	1 week
2011-03-17 11:25:32 +00:00
Jilles Tjoelker
0cf90cd1d3 bin: Prefer strrchr() to rindex().
This removes the last index/rindex usage from /bin.
2011-03-15 22:22:11 +00:00
Jilles Tjoelker
35c641ed21 sh: Fix some parameter expansion variants ${#...}.
These already worked: $# ${#} ${##} ${#-} ${#?}
These now work as well: ${#+word} ${#-word} ${##word} ${#%word}

There is an ambiguity in the standard with ${#?}: it could be the length of
$? or it could be $# giving an error in the (impossible) case that it is not
set. We continue to use the former interpretation as it seems more useful.
2011-03-13 20:02:39 +00:00
Stefan Farfeleder
5db2bbd692 Remove unnecessary cast.
Reviewed by:	jilles
2011-03-07 07:31:15 +00:00
Jilles Tjoelker
ea381e691a sh(1): Reduce excessive semicolon-separated sentences.
Reported by:	Benjamin Kaduk
2011-03-06 21:20:53 +00:00
Edward Tomasz Napierala
7123f4cd6f Export login class information via kinfo and make it possible to view
it using "ps -o class".
2011-03-05 14:41:49 +00:00
Jilles Tjoelker
976018d24f sh: Fix some warnings in code for arithmetic expressions.
Submitted by:	eadler
2011-03-05 13:27:13 +00:00
Jilles Tjoelker
3658c9776e kill: Note that this is used both as a normal program and a shell builtin. 2011-03-01 21:48:22 +00:00
Xin LI
eaea8924ce Accept == as an alias of = which is a popular GNU extension.
This is intentionally undocumented for now since it's not part
of any standard.

MFC after:	1 month
2011-02-27 12:28:06 +00:00
Hajimu UMEMOTO
35bb448cd6 When WITH_ICONV is set, use our in-tree iconv. 2011-02-26 18:54:54 +00:00
Sergey Kandaurov
440c834c54 mdoc(7) markup.
Approved by:	avg (mentor), kib (mentor)
MFC after:	3 days
2011-02-21 16:03:39 +00:00
Rebecca Cran
6bccea7c2b Fix typos - remove duplicate "the".
PR:	bin/154928
Submitted by:	Eitan Adler <lists at eitanadler.com>
MFC after: 	3 days
2011-02-21 09:01:34 +00:00
Jilles Tjoelker
2fd7d6aab6 test: Note that this is used both as a normal program and a shell builtin.
MFC after:	1 week
2011-02-15 22:17:47 +00:00
Jilles Tjoelker
e9749129ad sh: Detect dividing the smallest integer by -1.
This overflows and on some architectures such as amd64 it generates SIGFPE.
Generate an error on all architectures.
2011-02-12 23:44:05 +00:00
Rebecca Cran
f68fbfedc1 Fix typos.
PR:	docs/131625
Submitted by:	Andrew Wright <andrew at qemg.org>
MFC after:	1 month
2011-02-12 20:28:15 +00:00
Jilles Tjoelker
075b72ef01 sh(1): Update description of arithmetic. 2011-02-08 23:19:40 +00:00
Jilles Tjoelker
6262b84eee sh: Import arithmetic expression code from dash.
New features:
* proper lazy evaluation of || and &&
* ?: ternary operator
* executable is considerably smaller (8K on i386) because lex and yacc are
  no longer used

Differences from dash:
* arith_t instead of intmax_t
* imaxdiv() not used
* unset or null variables default to 0
* let/exp builtin (undocumented, will probably be removed later)

Obtained from:	dash
2011-02-08 23:18:06 +00:00
Jilles Tjoelker
b15e9aa322 sh: Fix two things about {(...)} <redir:
* In {(...) <redir1;} <redir2, do not drop redir1.
* Maintain the difference between (...) <redir and {(...)} <redir:
  In (...) <redir, the redirection is performed in the child, while in
  {(...)} <redir it should be performed in the parent (like {(...); :;}
  <redir)
2011-02-05 15:02:19 +00:00
Jilles Tjoelker
c059d82290 sh: Remove clearcmdentry()'s now unused argument. 2011-02-05 14:08:51 +00:00
Jilles Tjoelker
ef0cb80dd4 sh: Forget all cached command locations on any PATH change.
POSIX requires this and it is simpler than the previous code that remembered
command locations when appending directories to PATH.

In particular,
  PATH=$PATH
is no longer a no-op but discards all cached command locations.
2011-02-05 14:01:46 +00:00
Jilles Tjoelker
604e8224f8 sh: Do not try to execute binary files as scripts.
If execve() returns an [ENOEXEC] error, check if the file is binary before
trying to execute it using sh. A file is considered binary if at least one
of the first 256 bytes is '\0'.

In particular, trying to execute ELF binaries for the wrong architecture now
fails with an "Exec format error" message instead of syntax errors and
potentially strange results.
2011-02-05 12:54:59 +00:00
Jilles Tjoelker
3835f47c7e sh: Remove special code for shell scripts without magic number.
These are called "shell procedures" in the source.

If execve() failed with [ENOEXEC], the shell would reinitialize itself
and execute the program as a script. This requires a fair amount of code
which is not frequently used (most scripts have a #! magic number).
Therefore just execute a new instance of sh (_PATH_BSHELL) to run the
script.
2011-02-04 22:47:55 +00:00
Jilles Tjoelker
12dacf622b Make sys_signame upper case.
This matches the constants from <signal.h> with 'SIG' removed, which POSIX
requires kill and trap to accept and 'kill -l' to write.

'kill -l', 'trap', 'trap -l' output is now upper case.

In Turkish locales, signal names with an upper case 'I' are now accepted,
while signal names with a lower case 'i' are no longer accepted, and the
output of 'killall -l' now contains proper capital 'I' without dot instead
of a dotted capital 'I'.
2011-02-04 16:40:50 +00:00
Jilles Tjoelker
834d160b3a sh: Return only 126 or 127 for execve() failures.
Do not return 2 for errors other than [EACCES] or [ENOENT].
2011-02-03 23:38:11 +00:00
Jilles Tjoelker
3e0b768c63 sh: Remove comment mentioning herefd, which is gone. 2011-02-02 21:48:53 +00:00
Jilles Tjoelker
b9f696953d sh: Send messages about signals to stderr.
This is required by POSIX and seems to make more sense.

See also r217557.
2011-01-30 22:57:52 +00:00
Jilles Tjoelker
cff1d84937 sh: Clean up some old comments:
* There is no plan for an alternative to the command "set".
* Attempting to unset a readonly variable has not raised an error for quite
  a while, so the order of unsetting a variable and a function with the same
  name does not matter.

MFC after:	1 week
2011-01-25 20:56:18 +00:00
Konstantin Belousov
f5f3977b8d Document P_FOLLOWFORK.
MFC after:	2 weeks
2011-01-25 11:04:16 +00:00
Jilles Tjoelker
0d5ccb45d8 sh: Fix signal messages being sent to the wrong file sometimes.
When a foreground job exits on a signal, a message is printed to stdout
about this. The buffer was not flushed after this which could result in the
message being written to the wrong file if the next command was a builtin
and had stdout redirected.

Example:
  sh -c 'kill -9 $$'; : > foo; echo FOO:; cat foo

Reported by:	gcooper
MFC after:	1 week
2011-01-18 21:18:31 +00:00
Jilles Tjoelker
421fb02139 sh(1): Document changes to 'exit' from traps. 2011-01-16 14:11:50 +00:00
Jilles Tjoelker
ebdfd6dc4d sh: If exit is used without args from a trap action, exit on the signal.
This is useful so that it is easier to exit on a signal than to reset the
trap to default and resend the signal. It matches ksh93. POSIX says that
'exit' without args from a trap action uses the exit status from the last
command before the trap, which is different from 'exit $?' and matches this
if the previous command is assumed to have exited on the signal.

If the signal is SIGSTOP, SIGTSTP, SIGTTIN or SIGTTOU, or if the default
action for the signal is to ignore it, a normal _exit(2) is done with exit
status 128+signal_number.
2011-01-16 13:56:41 +00:00
Jilles Tjoelker
a043cc4c68 sh: Fix some things about -- in trap:
* Make 'trap --' do the same as 'trap' instead of nothing.
* Make '--' stop option processing (note that '-' action is not an option).

Side effect: The error message for an unknown option is different.
2011-01-15 21:09:00 +00:00
Jilles Tjoelker
45b3c17647 sh: Make 'trap -l' look like 'kill -l'. 2011-01-14 21:30:27 +00:00
Jilles Tjoelker
33a8413363 sh: Follow-up to r216743, grabstackblock() can be replaced with stalloc().
grabstackblock() was used only once (but it is a very often executed piece
of code).
2011-01-09 22:47:58 +00:00
Jilles Tjoelker
4b45b49a70 sh: Remove special %builtin PATH entry.
All builtins are now always found before a PATH search.

Most ash derivatives have an undocumented feature where the presence of an
entry "%builtin" in $PATH will cause builtins to be checked at that point of
the PATH search, rather than before looking at any directories as documented
in the man page (very old versions do document this feature).

I am removing this feature from sh, as it complicates the code, may violate
expectations (for example, /usr/bin/alias is very close to a forkbomb with
PATH=/usr/bin:%builtin, only /usr/bin/builtin not being another link saves
it) and appears to be unused (all the %builtin google code search finds is
in some sort of ash source code).

Note that aliases and functions took and take precedence above builtins.
Because aliases work on a lexical level they can only ever be overridden on
a lexical level (quoting or preceding 'builtin' or 'command'). Allowing
override of functions via PATH does not really fit in the model of sh and it
would work differently from %builtin if implemented.

Note: POSIX says special builtins are found before functions. We comply to
this because we do not allow functions with the same name as a special
builtin.

Silence from:	freebsd-hackers@ (message sent 20101225)
Discussed with:	dougb
2011-01-09 21:07:30 +00:00
Konstantin Belousov
50a57dfbec Move repeated MAXSLP definition from machine/vmparam.h to sys/vmmeter.h.
Update the outdated comments describing MAXSLP and the process
selection algorithm for swap out.

Comments wording and reviewed by:	alc
2011-01-09 12:50:44 +00:00
Jilles Tjoelker
70df11eaad sh: Make exit without parameters from EXIT trap POSIX-compliant.
It should use the original exit status, just like falling off the
end of the trap handler.

Outside an EXIT trap, 'exit' is still equivalent to 'exit $?'.
2011-01-08 23:08:13 +00:00
Jilles Tjoelker
e23a66ac83 sh: Do not call exitshell() from evalcommand() unless evalcommand() forked
itself.

This ensures that certain traps caused by builtins are executed.
2011-01-05 23:17:29 +00:00
Jaakko Heinonen
b38dc7ebea Increase carried_error if we skip a file due to an error. This ensures
that setfacl(1) exits with proper exit status on failure.

PR:		bin/149780
Submitted by:	Ævar Arnfjörð Bjarmason (original version)
Reviewed by:	trasz
MFC after:	3 weeks
2011-01-03 17:17:31 +00:00
Jilles Tjoelker
850460c0f1 sh: Check readonly status for assignments on regular builtins.
An error message is written, the builtin is not executed, nonzero exit
status is returned but the shell does not abort.

This was already checked for special builtins and external commands, with
the same consequences except that the shell aborts for special builtins.

Obtained from:	NetBSD
2011-01-01 13:26:18 +00:00
Jilles Tjoelker
09683f46b9 sh: Check if dup2 for redirection from/to a file succeeds.
A failure (e.g. caused by ulimit -n being set very low) is a redirection
error.

Example:
  ulimit -n 9; exec 9<.
2010-12-31 18:20:17 +00:00
Jilles Tjoelker
11535bdf04 sh: Avoid side effects from builtins in optimized command substitution.
Change the criterion for builtins to be safe to execute in the same process
in optimized command substitution from a blacklist of only cd, . and eval to
a whitelist.

This avoids clobbering the main shell environment such as by $(exit 4) and
$(set -x).

The builtins jobid, jobs, times and trap can still show information not
available in a child process; this is deliberately permitted. (Changing
traps is not.)

For some builtins, whether they are safe depends on the arguments passed to
them. Some of these are always considered unsafe to keep things simple; this
only harms efficiency a little in the rare case they are used alone in a
command substitution.
2010-12-30 22:33:55 +00:00
Jilles Tjoelker
685a270543 sh: Properly restore exception handler in fc.
If SIGINT arrived at exactly the right moment (unlikely), an exception
handler in a no longer active stack frame would be called.

Because the old handler was not used in the normal path, clang thought it
was a dead value and if an exception happened it would longjmp() to garbage.
This caused builtins/fc1.0 to fail if histedit.c was compiled with clang.

MFC after:	1 week
2010-12-29 19:39:51 +00:00
Jilles Tjoelker
acd7984f96 sh: Don't do optimized command substitution if expansions have side effects.
Before considering to execute a command substitution in the same process,
check if any of the expansions may have a side effect; if so, execute it in
a new process just like happens if it is not a single simple command.

Although the check happens at run time, it is a static check that does not
depend on current state. It is triggered by:
- expanding $! (which may cause the job to be remembered)
- ${var=value} default value assignment
- assignment operators in arithmetic
- parameter substitutions in arithmetic except ${#param}, $$, $# and $?
- command substitutions in arithmetic

This means that $((v+1)) does not prevent optimized command substitution,
whereas $(($v+1)) does, because $v might expand to something containing
assignment operators.

Scripts should not depend on these exact details for correctness. It is also
imaginable to have the shell fork if and when a side effect is encountered
or to create a new temporary namespace for variables.

Due to the $! change, the construct $(jobs $!) no longer works. The value of
$! should be stored in a variable outside command substitution first.
2010-12-28 21:27:08 +00:00
Jilles Tjoelker
45b71cd16e sh: Make expansion errors in optimized command substitution non-fatal.
Command substitutions consisting of a single simple command are executed in
the main shell process but this should be invisible apart from performance
and very few exceptions such as $(trap).
2010-12-28 13:28:24 +00:00
Jilles Tjoelker
ff802dc7bb sh: Simplify "stack string" code slightly.
Maintain a pointer to the end of the stack string area instead of how much
space is left. This simplifies the macros in memalloc.h. The places where
the new variable must be updated are only where the memory area is created,
destroyed or resized.
2010-12-27 22:18:27 +00:00
Jilles Tjoelker
78962f36d2 sh: Fix integer overflow check, it checked an uninitialized variable. 2010-12-26 13:41:53 +00:00
Jilles Tjoelker
d8f32e7287 sh: Allow arbitrary large numbers in CHECKSTRSPACE.
Reduce "stack string" API somewhat and simplify code.
Add a check for integer overflow of the "stack string" length (probably
incomplete).
2010-12-26 13:25:47 +00:00
Jilles Tjoelker
12dfb7a554 sh(1): Explain why it is a bad idea to use aliases in scripts. 2010-12-21 22:48:56 +00:00
Jilles Tjoelker
0a62a9caa9 sh: Add kill builtin.
This allows specifying a %job (which is equivalent to the corresponding
process group).

Additionally, it improves reliability of kill from sh in high-load
situations and ensures "kill" finds the correct utility regardless of PATH,
as required by POSIX (unless the undocumented %builtin mechanism is used).

Side effect: fatal errors (any error other than kill(2) failure) now return
exit status 2 instead of 1. (This is consistent with other sh builtins, but
not in NetBSD.)

Code size increases about 1K on i386.

Obtained from:	NetBSD
2010-12-21 22:47:34 +00:00
Jilles Tjoelker
5fe9123ff5 sh: Add a function to print warnings (with command name and newline).
This is like error() but without raising an exception.
It is particularly useful as a replacement for the warnx macro in
bltin/bltin.h.
2010-12-21 20:47:06 +00:00
Jilles Tjoelker
6a6760db7f sh: Make warnings in the printf builtin non-fatal, like in the program.
The #define for warnx now behaves much like the libc function (except that
it uses sh command name and output).

Also, it now uses C99 __VA_ARGS__ so there is no need for three different
macros for 0, 1 or 2 parameters.
2010-12-20 23:06:57 +00:00
Jilles Tjoelker
79357531c8 sh: arith: Disallow decimal constants starting with 0 (containing 8 or 9).
Constants in arithmetic starting with 0 should be octal only.

This avoids the following highly puzzling result:
  $ echo $((018-017))
  3
by making it an error instead.
2010-12-18 23:03:51 +00:00
Ulrich Spörlein
f6b767b026 Remove dead code.
c is assigned 0 and *loc is pointing to NULL, so c!=0 cannot be true,
and dereferencing loc would be a bad idea anyway.

Coverity Prevent:	CID 5113
Reviewed by:		jilles
2010-12-18 22:16:15 +00:00
Jilles Tjoelker
fa0951d63a sh: Fix corruption of command substitutions with special chars after newline
The CTLESC byte to protect a special character was output before instead of
after a newline directly preceding the special character.

The special handling of newlines is because command substitutions discard
all trailing newlines.
2010-12-16 23:28:20 +00:00
Ulrich Spörlein
326b41010a Remove duplicate check, turning dead code into live code.
Coverity CID:	5114
Reviewed by:	jilles
2010-12-13 10:48:49 +00:00
Jilles Tjoelker
b036c75b4c sh: Various simplifications to jobs.c:
* Prefer kill(-X) to killpg(X).
* Remove some dead code.
* No additional SIGINT is needed if int_pending() is already true.

No functional change is intended.
2010-12-12 22:59:34 +00:00
Jilles Tjoelker
9f5a68a002 sh: Remove the herefd hack.
The herefd hack wrote out partial here documents while expanding them. It
seems unnecessary complication given that other expansions just allocate
memory. It causes bugs because the stack is also used for intermediate
results such as arithmetic expressions. Such places should disable herefd
for the duration but not all of them do, and I prefer removing the need for
disabling herefd to disabling it everywhere needed.

Here documents larger than 1024 bytes will use a bit more CPU time and
memory.

Additionally this allows a later change to expand here documents in the
current shell environment. (This is faster for small here documents but also
changes behaviour.)

Obtained from:	dash
2010-12-12 00:07:27 +00:00
Jilles Tjoelker
f7dea8517f sh: Replace some macros and repeated code in expand.c with functions.
No functional change is intended, but the binary is about 1K smaller on
i386.
2010-12-11 22:13:29 +00:00
Jilles Tjoelker
6903c6832e sh: Use vsnprintf() rather than crafting our own in fmtstr().
Add INTOFF/INTON as longjmp out of vsnprintf may cause memory leaks or
undefined behaviour.
2010-12-11 17:47:27 +00:00
Jilles Tjoelker
c67712a089 sh: Improve internal-representation-to-text code to avoid binary output.
The code to translate the internal representation to text did not know about
various additions to the internal representation since the original ash and
therefore wrote binary stuff to the terminal.

The code is used in the jobs command and similar output.

Note that the output is far from complete and mostly serves for recognition
purposes.
2010-12-06 23:49:27 +00:00
Jilles Tjoelker
fa9e5d05a3 sh: POSIX says there should not be a space between Done and (exitstatus).
(On the other hand, (core dumped) does need a space and so does [1] +.)
2010-12-05 22:56:46 +00:00
Jilles Tjoelker
1bb49f9524 sh: Improve jobs output of pipelines.
If describing the status of a pipeline, write all elements of the pipeline
and show the status of the last process (which would also end up in $?).
Only write one report per job, not one for every process that exits.

To keep some earlier behaviour, if any process started by the shell in a
foreground job terminates because of a signal, write a message about the
signal (at most one message per job, however).

Also, do not write messages about signals in the wait builtin in
non-interactive shells. Only true foreground jobs now write such messages
(for example, "Terminated").
2010-12-05 22:37:01 +00:00
Jilles Tjoelker
ff304d3732 sh: Avoid marking a job as done before it is fully created.
In r208489, I added code to reap zombies when forking new processes, to
limit the amount of zombies. However, this can lead to marking a job as done
or stopped if it consists of multiple processes and the first process ends
very quickly. Fix this by only checking for zombies before forking the first
process of a job and not marking any jobs without processes as done or
stopped.
2010-12-05 21:53:29 +00:00
Jilles Tjoelker
5af61b5251 sh: jobs -p: Do not ask the kernel for the pgid.
The getpgid() call will fail if the first process in the job has already
terminated, resulting in output of "-1".

The pgid of a job is always the pid of the first process in the job and
other code already relies on this.
2010-12-05 16:09:03 +00:00
Ulrich Spörlein
cded07a878 Move most of the remaining USD/PSD/SMM papers into share/doc 2010-12-04 10:11:20 +00:00
Jilles Tjoelker
25f6b31fac sh(1): Clean up documentation of built-in commands.
Make sure all built-in commands are in the subsection named such, except
exp, let and wordexp which are deliberately undocumented. The text said only
built-ins that really need to be a built-in were documented there but in
fact almost all of them were already documented.
2010-12-03 23:24:27 +00:00
Jilles Tjoelker
b97d13b399 sh(1): Document that command's -p option also works with -v/-V.
This was implemented in r201343.
2010-12-01 23:26:32 +00:00
Jilles Tjoelker
9d37e15722 sh: Code size optimizations to "stack string" memory allocation:
* Prefer one CHECKSTRSPACE with multiple USTPUTC to multiple STPUTC.
* Add STPUTS macro (based on function) and use it instead of loops that add
  nul-terminated strings to the stack string.

No functional change is intended, but code size is about 1K less on i386.
2010-11-23 22:17:39 +00:00
Jilles Tjoelker
0bee28331e sh: Pass multiple bytes at a time to lex.
This speeds up the expansion/arith6.0 test considerably.
2010-11-23 20:46:06 +00:00
Jilles Tjoelker
ae7f47355d sh: Fix confusing behaviour if chdir succeeded but getcwd failed in cd -P.
If getcwd fails, do not treat this as an error, but print a warning and
unset PWD. This is similar to the behaviour when starting the shell in a
directory whose name cannot be determined.
2010-11-22 23:49:06 +00:00
Rebecca Cran
1161d4202c Fix some more warnings found by clang. 2010-11-22 20:10:48 +00:00
Jilles Tjoelker
467fdf32f8 sh: Remove the check that alpha/name/in_name chars are not CTL* bytes.
Since is_alpha/is_name/is_in_name were made ASCII-only, this can no longer
happen.

Additionally, the check was wrong because it did not include the new
CTLQUOTEEND.
2010-11-20 14:30:28 +00:00
Jilles Tjoelker
aeb5d06504 sh: Code size optimizations to buffered output.
This is mainly less use of the outc macro.

No functional change is intended, but code size is about 2K less on i386.
2010-11-20 14:14:52 +00:00
Jilles Tjoelker
9897c45f31 sh: Add printf builtin.
This was removed in 2001 but I think it is appropriate to add it back:
* I do not want to encourage people to write fragile and non-portable echo
  commands by making printf much slower than echo.
* Recent versions of Autoconf use it a lot.
* Almost no software still wants to support systems that do not have
  printf(1) at all.
* In many other shells printf is already a builtin.

Side effect: printf is now always the builtin version (which behaves
identically to /usr/bin/printf) and cannot be overridden via PATH (except
via the undocumented %builtin mechanism).

Code size increases about 5K on i386. Embedded folks might want to replace
/usr/bin/printf with a hard link to /usr/bin/alias.
2010-11-19 12:56:13 +00:00
Jilles Tjoelker
c3f57269e6 sh: Add binary buffered output for use by the printf builtin. 2010-11-14 15:31:59 +00:00
Jilles Tjoelker
d79326ecc8 sh: Update the suspend example for the change of the job control flag
from -j to -m, many years ago.

Due to r215266, this function now actually works.
2010-11-13 22:20:46 +00:00
Jilles Tjoelker
4a7b1013fb sh: Do the additional actions if 'local -' restore changes -i/-m/-E/-V.
Example:
  f() { local -; set +m; }; f
caused failure to execute external programs because the job control tty fd
was not opened.
2010-11-13 22:10:26 +00:00
Jilles Tjoelker
61aad2fe09 echo(1): Clarify portability and mention literal "--" handling.
MFC after:	1 week
2010-11-12 22:43:40 +00:00
Jilles Tjoelker
4b985a89e7 sh(1): Document r214304 (special builtin is illegal function name). 2010-11-12 22:40:18 +00:00
Jilles Tjoelker
f35d74beed sh(1): Update for r214492. "${v+"hi}there"}".
The part hi}there is not a quoted string but nevertheless the closing brace
does not terminate the expansion.
2010-11-12 22:28:47 +00:00
Jilles Tjoelker
7f39c0011f sh: Remove unused man page for echo builtin.
The information in sh(1) about the echo builtin is equivalent, though less
extensive.

The echo(1) man page (bin/echo/echo.1) is different.

Unfortunately, sh's echo builtin and /bin/echo have gone out of sync and
this probably cannot be fixed any more.

Reported by:	uqs (list of untouched files)
MFC after:	1 week
2010-11-12 15:40:00 +00:00
Jilles Tjoelker
3fdfd0a435 sh(1): Modernize the introduction a bit.
In particular, remove the text about ksh-like features, which are usually
taken for granted nowadays. The original Bourne shell is fading away and for
most users our /bin/sh is one of the most minimalistic they know.
2010-11-12 14:40:20 +00:00
Jilles Tjoelker
47fdf870a7 test: Move tests to tools/regression/bin/test.
Convert the tests to the perl prove format.
Remove obsolete TEST.README (results of an old TEST.sh for some old Unices)
and TEST.csh (old tests without correct values, far less complete than
TEST.sh).

MFC after:	1 week
2010-11-08 23:15:10 +00:00
Jilles Tjoelker
135ff4b5b0 sh: Fix some issues with aliases and case, by importing dash checkkwd code.
This moves the function of the noaliases variable into the checkkwd
variable. This way it is properly reset on errors and aliases can be used
normally in the commands for each case (the case labels recognize the
keyword esac but no aliases).

The new code is clearer as well.

Obtained from:	dash
2010-11-02 23:44:29 +00:00
Ed Schouten
736fc28680 Add a new libc function: cfmakesane(3).
I've noticed various terminal emulators that need to obtain a sane
default termios structure use very complex `hacks'. Even though POSIX
doesn't provide any functionality for this, extend our termios API with
cfmakesane(3), which is similar to the commonly supported cfmakeraw(3),
except that it fills the termios structure with sane defaults.

Change all code in our base system to use this function, instead of
depending on <sys/ttydefaults.h> to provide TTYDEF_*.
2010-11-02 17:00:56 +00:00
Jilles Tjoelker
57a40f7d08 sh(1): Correct synopsis and make precise how $0 is set.
In particular, the extra argument to set $0 with -c was not documented.

MFC after:	1 week
2010-10-31 23:03:11 +00:00
Ulrich Spörlein
ec1e236534 Fix typo and grammar nit
Submitted by:	arundel
MFC after:	7 days (or when the bikeshed has abated)
2010-10-31 19:16:54 +00:00
Jilles Tjoelker
4c4164f9a7 sh: Reindent evaltree(). 2010-10-31 12:08:16 +00:00
Jilles Tjoelker
dca867f1c9 sh: Use iteration instead of recursion to evaluate semicolon lists.
This reduces CPU and memory usage when executing long lists (such
as long functions).
2010-10-31 12:06:02 +00:00
Ulrich Spörlein
c2c9a54067 Elaborate some more on the non-security implications of using -P
Submitted by:	delphij
Discussion at:	svn-src-all
2010-10-31 09:21:27 +00:00
Jilles Tjoelker
274110df0a sh: Tweak some string constants to reduce code size.
* Reduce some needless differences.
* Shorten some error messages that should not happen.
2010-10-29 21:44:43 +00:00
Jilles Tjoelker
a1251487f4 sh: Reject function names ending in one of !%*+-=?@}~
These do something else in ksh: name=(...) is an array or compound variable
assignment and the others are extended patterns.

This is the last patch of the ones tested in the exp run.

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-29 21:20:56 +00:00
Jilles Tjoelker
e20776d503 sh: Detect various additional errors in the parser.
Apart from detecting breakage earlier or at all, this also fixes a segfault
in the testsuite. The "handling" of the breakage left an invalid internal
representation in some cases.

Examples:
  echo a; do echo b
  echo `) echo a`
  echo `date; do do do`

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-29 21:06:57 +00:00
Jilles Tjoelker
33582ce055 sh: Error out on various specials/keywords in the wrong place in backticks.
Example:
  echo `date)`

Exp-run done by:	pav (with some other sh(1) changes)
Obtained from:		NetBSD (Christos Zoulas, NetBSD PR 11317)
2010-10-29 20:23:41 +00:00
Jilles Tjoelker
60f7eec450 sh: Fix some issues with CTL* bytes and ${var#pat}.
subevalvar() incorrectly assumed that CTLESC bytes were present iff the
expansion was quoted. However, they are present iff various processing such
as word splitting is to be done later on.

Example:
  v=@$e@$e@$e@
  y="${v##*"$e"}"
  echo "$y"
failed if $e contained the magic CTLESC byte.

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-29 19:34:57 +00:00
Jilles Tjoelker
048f26671a sh: Do IFS splitting on word in ${v+word} and ${v-word}.
The code is inspired by NetBSD sh somewhat, but different because we
preserve the old Almquist/Bourne/Korn ability to have an unquoted part in a
quoted ${v+word}. For example, "${v-"*"}" expands to $v as a single field if
v is set, but generates filenames otherwise.

Note that this is the only place where we split text literally from the
script (the similar ${v=word} assigns to v and then expands $v). The parser
must now add additional markers to allow the expansion code to know whether
arbitrary characters in substitutions are quoted.

Example:
  for i in ${$+a b c}; do echo $i; done

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-29 13:42:18 +00:00
Jilles Tjoelker
6c38071288 sh: Only accept a '}' inside ${v+-=?...} if double-quote state matches.
If double-quote state does not match, treat the '}' literally.

This ensures double-quote state remains the same before and after a
${v+-=?...} which helps with expand.c.

It makes things like
  ${foo+"\${bar}"}
which I have seen in the wild work as expected.

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-28 22:34:49 +00:00
Jilles Tjoelker
9cec947f3f sh: Make double-quotes quote a '}' inside ${v#...} and ${v%...}.
Exp-run done by:	pav (with some other sh(1) changes)
PR:			bin/57554
2010-10-28 21:51:14 +00:00
Dag-Erling Smørgrav
1f9d174116 Language cleanup. 2010-10-27 18:48:23 +00:00
Jilles Tjoelker
d94c867339 sh: Ignore double-quotes in arithmetic rather than treating them as quotes.
This provides similar behaviour, but allows a simpler parser.

This changes r206473.

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-24 22:25:38 +00:00
Jilles Tjoelker
67e109adbe sh: Do not allow overriding a special builtin with a function.
This is a syntax error.

POSIX does not say explicitly whether defining a function with the same name
as a special builtin is allowed, but it does say that it is impossible to
call such a function.

A special builtin can still be overridden with an alias.

This commit is part of a set of changes that will ensure that when
something looks like a special builtin to the parser, it is one. (Not the
other way around, as it remains possible to call a special builtin named
by a variable or other substitution.)

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-24 22:03:21 +00:00
Jilles Tjoelker
074e83b14e sh: Make sure defined functions can actually be called.
Add some conservative checks on function names:
- Disallow expansions or quoting characters; these can only be called via
  strange control characters
- Disallow '/'; these functions cannot be called anyway, as exec.c assumes
  they are pathnames
- Make the CTL* bytes work properly in function names.

These are syntax errors.

POSIX does not require us to support more than names (letters, digits and
underscores, not starting with a digit), but I do not want to restrict it
that much at this time.

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-24 20:45:13 +00:00
Jilles Tjoelker
3dec7d0c15 sh: Check whether dup2 was successful for >&FD and <&FD.
A failure (usually caused by FD not being open) is a redirection error.

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-24 20:09:49 +00:00
Jilles Tjoelker
ba08f69b5c sh: Change ! within a pipeline to start a new pipeline instead.
This is how ksh93 treats ! within a pipeline and makes the ! in
  a | ! b | c
negate the exit status of the pipeline, as if it were
  a | { ! b | c; }

Side effect: something like
  f() ! a
is now a syntax error, because a function definition takes a command,
not a pipeline.

Exp-run done by:	pav (with some other sh(1) changes)
2010-10-24 17:06:49 +00:00
Jilles Tjoelker
f1ec058177 sh(1): Clarify subshells/processes for pipelines.
For multi-command pipelines,
1. all commands are direct children of the shell (unlike the original
   Bourne shell)
2. all commands are executed in a subshell (unlike the real Korn shell)

MFC after:	1 week
2010-10-16 14:37:56 +00:00
Jilles Tjoelker
9fa5f4a093 sh: Use <stddef.h> rather than <sys/stddef.h>.
<sys/stddef.h> is only for the kernel and conflicts with <stddef.h>.
2010-10-16 12:40:00 +00:00
David E. O'Brien
56d47fb9b6 We only need to look as far as '..' to find 'test/'. 2010-10-13 23:31:17 +00:00
David E. O'Brien
7cfe69417c Do not assume in growstackstr() that a "precious" character will be
immediately written into the stack after the call.  Instead let the caller
manage the "space left".

Previously, growstackstr()'s assumption causes problems with STACKSTRNUL()
where we want to be able to turn a stack into a C string, and later
pretend the NUL is not there.

This fixes a bug in STACKSTRNUL() (that grew the stack) where:
1. STADJUST() called after a STACKSTRNUL() results in an improper adjust.
   This can be seen in ${var%pattern} and ${var%%pattern} evaluation.
2. Memory leak in STPUTC() called after a STACKSTRNUL().

Reviewed by:	jilles
2010-10-13 23:29:09 +00:00
David E. O'Brien
8832864298 In the spirit of r90111, depend on c89 and remove the "STATIC" macro
and its usage.
2010-10-13 22:18:03 +00:00
David E. O'Brien
f10d20060e If one wishes to set breakpoints of static the functions here, they
cannot be inlined.

Submitted by:	jhb
2010-10-13 18:23:43 +00:00
John Baldwin
8ab2e97063 Make DEBUG traces 64-bit clean:
- Use %t to print ptrdiff_t values.
- Cast a ptrdiff_t value explicitly to int for a field width specifier.

While here, sort includes.

Submitted by:	Garrett Cooper
2010-10-13 13:22:11 +00:00
John Baldwin
f12f3dbeee Suggest that DEBUG_FLAGS be used to enable extra debugging rather than
frobbing CFLAGS directly.  DEBUG_FLAGS is something that can be specified
on the make command line without having to edit the Makefile directly.

Submitted by:	Garrett Cooper
2010-10-13 13:17:38 +00:00
David E. O'Brien
aa7b6f8259 Consistently use "STATIC" for all functions in order to be able to set
breakpoints with in a debugger.  And use naked "static" for variables.

Noticed by:	bde
2010-10-13 04:01:01 +00:00
David E. O'Brien
9d22cd9be8 If DEBUG is 3 or greater, disable STATICization of functions.
Also correct the documented location of the trace file.
2010-10-12 19:24:41 +00:00
David E. O'Brien
f3bf9b7a16 Allow one to regression test 'sh' changes without having to install
a potentially bad /bin/sh first.
2010-10-12 18:20:38 +00:00
Ulrich Spörlein
a3800f8f0e rm(1): clarify that -P works only when blocks are updated in-place
Suggested by:	pjd, ivoras, arundel
2010-10-08 15:20:20 +00:00
Ulrich Spörlein
0d9deed52c mdoc: drop redundant .Pp and .LP calls
They have no effect when coming in pairs, or before .Bl/.Bd
2010-10-08 12:40:16 +00:00
Ed Maste
6239ef1d29 Make a thread's address available via the kern proc sysctl, just like the
process address.

Add "tdaddr" keyword to ps(1) to display this thread address.

Distilled from Sandvine's patch set by Mark Johnston.
2010-10-08 00:44:53 +00:00
Xin LI
873ddec3fb Clarify the combination effect of -P and -f to make it clear.
Submitted by:	arundel
MFC after:	2 weeks
2010-10-04 06:17:45 +00:00
Ed Maste
ce026da7e5 Correct sort order. 2010-09-22 19:26:41 +00:00
Warner Losh
9768ccae05 Merge from tbemd:
Add directory names directly and sort at the end.
Include bsd.arch.inc.mk so we can, in the future, more easily make arch
dependent changes in /bin (unlikely, but is needed for symmetry).
2010-09-13 01:29:51 +00:00
Jilles Tjoelker
2b11dfee8e sh: Add __dead2 to two functions that do not return.
Apart from helping static analyzers, this also appears to reduce the size of
the binary slightly.
2010-09-12 22:00:31 +00:00
Jilles Tjoelker
8f2dc7de67 sh: Fix exit status if return is used within a loop condition. 2010-09-11 15:07:40 +00:00
Jilles Tjoelker
011d162dd3 sh: Apply variable assignments left-to-right in bltinlookup().
Example:
  HOME=foo HOME=bar cd
2010-09-11 14:15:50 +00:00
Rebecca Cran
5512804bb8 Revert changes of 'assure' to 'ensure' made in r211936.
Approved by: rrs (mentor)
2010-09-11 10:49:56 +00:00
Jilles Tjoelker
c68e12a6f9 test(1): Fix markup, ( and ) must be separate arguments so leave spaces.
MFC after:	1 week
2010-09-10 14:03:58 +00:00
Jilles Tjoelker
2f67f12e04 test(1): Clarify grammar ambiguity and -a/-o vs shell &&/||. 2010-09-10 14:00:27 +00:00
Jilles Tjoelker
1217a4ead5 sh(1): Remove xrefs for expr(1) and getopt(1).
expr(1) should usually not be used as various forms of parameter expansion
and arithmetic expansion replicate most of its functionality in an easier
way.

getopt(1) should not be used at all in new code. Instead, getopts(1) or
entirely manual parsing should be used.

MFC after:	1 week
2010-09-10 13:40:31 +00:00
Jilles Tjoelker
f390d8d927 expr(1): Add sh(1) versions of examples, remove an incorrect example.
The three examples are better done using sh(1) itself these days.

The example
  expr -- "$a" : ".*"
is incorrect in the general case, as "$a" may be an operator.

MFC after:	2 weeks
2010-09-09 21:59:53 +00:00
Jilles Tjoelker
917fdfb106 sh: Fix 'read' if all chars before the first IFS char are backslash-escaped.
Backslash-escaped characters did not set the flag for a non-IFS character.

MFC after:	2 weeks
2010-09-08 20:35:43 +00:00
Jilles Tjoelker
2ca3d70fc6 sh: Improve comments in expand.c. 2010-09-05 21:12:48 +00:00
Jilles Tjoelker
27542743cd sh: Get rid of some magic numbers.
MFC after:	1 week
2010-09-04 21:23:46 +00:00
Jilles Tjoelker
fe5d61a4cf sh: Do not use locale for determining if something is a name.
This makes it impossible to use locale-specific characters in variable
names.

Names containing locale-specific characters make scripts only work with the
correct locale setting. Also, they did not even work in many practical cases
because multibyte character sets such as utf-8 are not supported.

This also avoids weirdness if LC_CTYPE is changed in the middle of a script.
2010-09-03 22:13:54 +00:00
Brian Somers
3b8199dbfc Touch the man page date after updating the ustar limitations.
MFC after:	3 weeks
2010-08-29 12:17:16 +00:00
Brian Somers
cd14b45756 Correct an out-by-one error when earlying out ustar filenames that
are too long.  Filenames escaping this test are caught later on,
so the bug doesn't cause any breakage.

Document the correct ustar limitations in pax.  As I have no access
to the IEEE 1003.2 spec, I can only assume that the limitations
imposed are in fact correct.

Add regression tests for the filename limitations imposed by pax.

MFC after:	3 weeks
2010-08-29 11:56:56 +00:00
Rebecca Cran
e7f8dd75b3 Fix incorrect usage of 'assure' and 'insure'.
Approved by: rrs (mentor)
2010-08-28 16:32:01 +00:00
Brian Somers
f77b6c6a25 Fix an off-by-one error where we try to split a path name that's
more than 100 characters long and the 101th last character is a
'/'.

MFC after:	3 weeks
2010-08-27 08:54:40 +00:00
Jilles Tjoelker
8fdbdb5d50 sh: Remove remnants of '!!' to negate pattern.
This Almquist extension was disabled long ago.

In pathname generation, components starting with '!!' were treated as
containing wildcards, causing unnecessary readdir (which could fail, causing
pathname generation to fail while it should not).
2010-08-22 21:18:21 +00:00
Jilles Tjoelker
36cf3efe41 sh(1): Add a brief summary of arithmetic expressions. 2010-08-22 13:04:00 +00:00
Jilles Tjoelker
ae7c0700dc sh: Fix break/continue/return sometimes not skipping the rest of dot script.
In our implementation and most others, a break or continue in a dot script
can break or continue a loop outside the dot script. This should cause all
further commands in the dot script to be skipped. However, cmdloop() did not
know about this and continued to parse and execute commands from the dot
script.

As described in the man page, a return in a dot script in a function returns
from the function, not only from the dot script. There was a similar issue
as with break and continue. In various other shells, the return appears to
return from the dot script, but POSIX seems not very clear about this.
2010-08-15 21:06:53 +00:00
Jilles Tjoelker
c5e4fa998d sh: Add a forgotten const. 2010-08-13 20:29:43 +00:00
Jilles Tjoelker
92378cce77 sh: Fix shadowing of sigset. 2010-08-13 13:36:18 +00:00
Jilles Tjoelker
c8a3d81f34 sh: Fix heap-based buffer overflow in pathname generation.
The buffer for generated pathnames could be too small in some cases. It
happened to be always at least PATH_MAX long, so there was never an overflow
if the resulting pathnames would be usable.

This bug may be abused if a script subjects input from an untrusted source
to pathname generation, which a bad idea anyhow. Most shell scripts do not
work on untrusted data. secteam@ says no advisory is necessary.

PR:		bin/148733
Reported by:	Changming Sun snnn119 at gmail com
MFC after:	10 days
2010-08-10 22:45:59 +00:00
Jilles Tjoelker
40969e7396 Remove unnecessary duplicate letters in mksyntax.c,
the table elements would just be overwritten twice.
2010-08-08 21:04:27 +00:00
Jilles Tjoelker
b84d7af7c6 sh: Return 0 from eval if no command was given.
This makes a difference if there is a command substitution.

To make this work, evalstring() has been changed to set exitstatus to 0 if
no command was executed (the string contained only whitespace).

Example:
  eval $(false); echo $?
should print 0.
2010-08-03 22:17:29 +00:00
Konstantin Belousov
67a54762a1 Document the new argument parser for sleep(1) and SIGINFO behaviour.
Remove higlight for the unportability notice.

MFC after:	3 weeks
2010-08-02 10:59:23 +00:00
Konstantin Belousov
f6e6dc63ab Simplify argument parser by using sscanf(); simplify usage() by not
refusing to use stdio.
Reduce nesting level in the sleep loop by returning earlier for negative
timeouts.
Limit the maximum timeout to INT_MAX seconds.

Submitted by:	bde
MFC after:	3 weeks
2010-08-02 10:57:56 +00:00
Jilles Tjoelker
933803fb21 sh: Do not enter consecutive duplicates into the history.
This simply sets a flag in libedit. It has a shortcoming in that it does not
apply to multi-line commands.

Note that a configuration option for this is not going to happen, but always
having this seems better than not having it. NetBSD has done the same.

PR:		bin/54683
Obtained from:	NetBSD
MFC after:	1 month
2010-08-01 16:37:51 +00:00
Joel Dahl
a53bb70bda Spelling fixes. 2010-07-31 21:09:49 +00:00
Konstantin Belousov
bc9bbbe02c Deal with proper format for printing time_t.
Reported by:	ache
MFC after:	3 weeks
2010-07-31 17:41:58 +00:00
Konstantin Belousov
13d8eedae7 Report the time left for the sleep on SIGINFO.
Be stricter in the checking of interval specification.

PR:	bin/139345
MFC after:	3 weeks
2010-07-31 14:30:11 +00:00
Jilles Tjoelker
9e7839f632 kill: Stop processing if a syntactically invalid pid is encountered.
So a command like
  kill _HUP 1
now fails without sending SIGTERM to init.

The behaviour when kill(2) fails remains unchanged: processing continues.
This matches other implementations and POSIX and is useful for killing
multiple processes at once when some of them may already be gone.

PR:		bin/40282
2010-07-29 16:40:45 +00:00
Benedict Reuschling
c73b6b8f1d Typo fix: s/ommited/omitted
MFC after: 4 days
2010-07-27 17:33:02 +00:00
Jilles Tjoelker
6c0c240366 sh: Fix crash due to uninitialized here-document.
If an ; or & token was followed by an EOF token, pending here-documents were
left uninitialized. Execution would crash, either in the main shell process
for literal here-documents or in a child process for expanded
here-documents. In the latter case the problem is hard to detect apart from
the core dumps and log messages.

Side effect: slightly different retries on inputs where EOF is not
persistent.

Note that tools/regression/bin/sh/parser/heredoc6.0 still causes a similar
crash in a child process. The text passed to eval is malformed and should be
rejected.
2010-07-25 22:25:52 +00:00
Edward Tomasz Napierala
ab8d4d579f Fix alignment for the 'flags' label, and make more room for 'tdev'.
MFC after:	1 month
2010-07-24 15:37:36 +00:00
Edward Tomasz Napierala
9cde081d0a Add P_HASTHREADS flag description. 2010-07-24 15:24:12 +00:00
Jilles Tjoelker
cac4830ce4 sh: Allow a background command consisting solely of redirections.
Example:
  </dev/null &

MFC after:	2 weeks
2010-07-18 12:45:31 +00:00
Jilles Tjoelker
d5af15eabf sh: There cannot be a TNOT in simplecmd(), remove checks.
simplecmd() only handles simple commands and function definitions, neither
of which involves the ! keyword. The initial token on entry to simplecmd()
is one of the following: TSEMI, TAND, TOR, TNL, TEOF, TWORD, TRP.
2010-07-14 22:31:45 +00:00
Brian Somers
b040210d7a Add -l to the synopsis
Submitted by:	jhell at dataix dot net
MFC after:	3 days
2010-07-12 01:58:46 +00:00
Jilles Tjoelker
c9c987cd35 sh: Use $PWD instead of getcwd() for the \w and \W prompt expansions.
This ensures that the logical working directory (which may include
symlinks) is shown and is similar to the default behaviour of the pwd
builtin.
2010-07-02 22:17:13 +00:00
Jilles Tjoelker
ed4c3b5f86 sh: Forget about terminated background processes sooner.
Unless $! has been referenced for a particular job or $! still contains that
job's pid, forget about it after it has terminated. If $! has been
referenced, remember the job until the wait builtin has reported its
completion (either with the pid as parameter or without parameters).

In interactive mode, jobs are forgotten after termination has been reported,
which happens before primary prompts and through the jobs builtin. Even
then, though, remember a job if $! has been referenced.

This is similar to what is suggested by POSIX and should fix most memory
leaks (which also tend to cause sh to use more CPU time) with long running
scripts that start background jobs.

Caveats:
* Repeatedly referencing $! without ever doing 'wait', like
    while :; do foo & echo started foo: $!; sleep 60; done
  will still use a lot of memory and CPU time in the long run.
* The jobs and jobid builtins do not cause a job to be remembered for longer
  like expanding $! does.

PR:		bin/55346
2010-06-29 22:37:45 +00:00
Brian Somers
3ba0e47047 Recognise the -l switch with pkill - list kill command(s) used.
PR:		143558
Submitted by:	eitanadlerlist at gmail dot com
MFC after:	3 weeks
2010-06-20 08:48:30 +00:00
Jilles Tjoelker
49e10f5e38 sh: Fix compilation with -DNO_HISTORY.
The LINENO code uses snprintf() and relied on "myhistedit.h" to pull in the
necessary <stdio.h>.

Compiling with -DNO_HISTORY disables all editing and history support and
allows linking without -ledit -ltermcap. This may be useful for embedded
systems.

MFC after:	2 weeks
2010-06-19 10:33:04 +00:00
Jilles Tjoelker
e46b12b732 sh: Add filename completion.
This uses the new libedit completion function with quoting support.

Unlike NetBSD, there is no 'set +o tabcomplete' option to disable
completion. I do not see any reason for such a special treatment, as
completion is rather useful and it is possible to do
  bind ^I ed-insert
to disable completion and insert a tab character instead.

Submitted by:	Guy Yur
2010-06-15 21:58:40 +00:00
Ulrich Spörlein
a90fbe1dbb More mdoc nitpicking to improve compatibility to mdocml
- .Nd in section NAME is not optional
- .Ed was missing
- "indent" is not a flag, but a literal argument for -offset
- stop switching font sizes for acronyms
- use .Brq instead of rolling our own
2010-06-11 06:04:07 +00:00
Jilles Tjoelker
f3d893fcde sh: Pass through SIGINT from a child if interactive and job control
is enabled.

This already worked if without job control.

In either case, this depends on it that a process that terminates due to
SIGINT exits on it (so not with status 1, or worse, 0).

Example:
  sleep 5; echo continued
This does not print "continued" any more if sleep is aborted via ctrl+c.

MFC after:	1 month
2010-06-06 22:27:32 +00:00
Edward Tomasz Napierala
9fa5f90fbf Don't try to copy a socket after "xxx is a socket (not copied)." message.
Previously, it would either try to copy it anyway and fail (without -R),
or create fifo instead of the socket (with -R).

Found with:	Coverity Prevent
CID:		5623
MFC after:	2 weeks
2010-06-05 08:50:39 +00:00
Brian Somers
5990bfb40b Explain that setting an absolute day value can fail and that the month should
usually be set first when using -v.

Adjust an example that sets the day to 30 before setting the month to 3 in
accordance with this approach as the example would always fail in February!

PR:		147354
MFC after:	2 weeks
2010-06-04 06:56:58 +00:00
Jilles Tjoelker
580eefdf4f sh: Pass TERM changes to libedit.
I have changed the patch slightly to ignore TERM changes in subshells.

PR:		bin/146916
Submitted by:	Guy Yur
Obtained from:	NetBSD
2010-06-02 19:16:58 +00:00
Ulrich Spörlein
c5d27505b6 Zero struct before reading from it
PR:	   140384
Found by:  clang static analyzer
MFC after: 3 weeks
2010-06-02 09:34:41 +00:00
Jilles Tjoelker
5d91007000 sh: Fix a crash if a heredoc was not properly ended and parsing continued.
Example (in interactive mode):
  cat <<EOF && )
The next command typed caused sh to segfault, because the state for the here
document was not reset.

Like parser_temp, this uses the fact that the parser is not re-entered.
2010-05-30 14:20:32 +00:00
Jilles Tjoelker
ba02a307fe sh: Change interaction of command substitution and here documents.
If a command substitution contains a newline token, this no longer starts
here documents of outer commands. This way, we follow POSIX's idea of the
command substitution being a separate script more closely. It also matches
other shells better and is consistent with newline characters in quotes not
starting here documents.

The extension tested in parser/heredoc3.0 ($(cat <<EOF)\ntext\nEOF\n)
continues to be supported.

In particular, this change allows things like
  cat <<EOF && echo `pwd`
(a `` command substitution after a here document)
which formerly silently used an empty file as the here document, because the
EOF of the inner command "pwd" also forced an empty here document.
2010-05-30 14:11:27 +00:00
Jilles Tjoelker
c1564db05d sh: Recognize "--" in . and exec.
Although "--" historically has not been required to be recognized for
certain special builtins that do not take options in POSIX, some other
implementations recognize options for them, requiring scripts to use "--" or
avoid operands starting with "-".

Operands starting with "-" can be avoided with eval by prepending a space,
and cannot occur with break, continue, exit, return and shift as they only
take numbers, nor with times as it does not take operands. With . and exec,
avoiding "-" is not so easy as it may require reimplementing the PATH
search; therefore the current proposal for POSIX is to require recognition
of "--" for them.

We continue to accept other strings starting with "-" as operands to . and
exec, and also "--" if it is alone to . (which would otherwise be invalid
anyway).
2010-05-28 22:40:24 +00:00
Jilles Tjoelker
1788b7ffb9 sh(1): Rework documentation of shell variables.
* Move the "environment variables" that do not need exporting to be
  effective or that are set by the shell without exporting to a new section
  "Special Variables".
* Add special variables LINENO and PPID.
* Add environment variables LANG, LC_* and PWD; also describe ENV under
  environment variables.
2010-05-24 15:12:12 +00:00
Jilles Tjoelker
626a7b1d90 sh(1): Improve wording of 'Special Parameters' section. 2010-05-24 13:28:12 +00:00
Jilles Tjoelker
1f65c541e8 sh: Reap any zombies before forking for a background command.
This prevents accumulating huge amounts of zombies if a script executes
many background commands but no external commands or subshells.

Note that zombies will not be reaped during long calculations (within
the shell process) or read builtins, but those actions do not create
more zombies.

The terminated background commands will also still be remembered by the
shell.

PR:		bin/55346
2010-05-24 10:35:57 +00:00
Ulrich Spörlein
c9022a35b2 Fix back references in substitute command for pax(1)
pax(1) was trying to copy the back-referenced data from
the match pattern, not the matched data.

PR:		bin/118132
Obtained from:	Debian bug #451361
Reviewed by:	jilles
MFC after:	3 weeks
2010-05-24 06:33:14 +00:00
Edward Tomasz Napierala
413f7daf59 Update authors and history.
MFC after:	1 week
2010-05-13 16:46:23 +00:00
Jilles Tjoelker
4710b07ecf sh: Fix pathname expansion with quoted slashes like *\/.
These are git commits 36f0fa8fcbc8c7b2b194addd29100fb40e73e4e9 and
d6d06ff5c2ea0fa44becc5ef4340e5f2f15073e4 in dash.

Because this is the first code I'm importing from dash to expand.c, add the
Herbert Xu copyright notice which is in dash's expand.c.

When pathname expanding *\/, the CTLESC representing the quoted state was
erroneously taken as part of the * pathname component. This CTLESC was then
seen by the pattern matching code as escaping the '\0' terminating the
string.

The code is slightly different because dash converts the CTLESC characters
to backslashes and removes all the other CTL* characters to allow
substituting glob(3).

The effect of the bug was also slightly different from dash (where nothing
matched at all). Because a CTLESC can escape a '\0' in some way, whether
files were included despite the bug depended on memory that should not be
read. In particular, on many machines /*\/ expanded to a strict subset of
what /*/ expanded to.

Example:
  echo /*"/null"

This should print /dev/null, not /*/null.

PR:		bin/146378
Obtained from:	dash
2010-05-11 23:19:28 +00:00