Commit Graph

1108 Commits

Author SHA1 Message Date
jilles
74ddfdb122 sh: Fix memory leak with an assignment before a regular builtin.
MFC after:	1 week
2014-03-27 22:57:23 +00:00
jilles
ce55d2a446 sh: Fix memory leak when trying to set a read only variable.
MFC after:	1 week
2014-03-27 22:52:26 +00:00
jilles
270892ce0a sh: Fix possible memory leaks and double frees with unexpected SIGINT. 2014-03-26 20:43:40 +00:00
jilles
ba6f930b2f sh: Don't overwrite old exit status if a PID is reused.
Only store exit status for a process if that process has not terminated yet.

Test (slow):
  exit 7 & p1=$!; until exit 8 & p2=$!; [ "$p1" = "$p2" ]; do wait "$p2";
  done; sleep 0.1; wait %1; echo $?
should write "7".
2014-03-20 22:38:13 +00:00
jilles
94dae0aff1 sh: Allow kill %job on jobs started without job control.
When killing a %job started without job control, kill all processes in it.
As with process groups and zombies, if any process in the job can be killed
or has already terminated, the command is successful.

This also fixes occasional failures of the builtins/kill1.0 test.
2014-03-15 14:58:48 +00:00
jilles
eeed81169f sh: Add some consts. 2014-03-14 21:45:37 +00:00
jmmv
3cbab1b573 Make bsd.test.mk the only public mk fragment for the building of tests.
Change {atf,plain,tap}.test.mk to be internal implementation details of
bsd.test.mk.  Makefiles that build tests should now only include bsd.test.mk
and declaratively specify what they want to build, without worrying about
the internal implementation of the mk files.

The reason for this change is to permit building test programs of different
interfaces from a single directory, which is something I had a need for
while porting tests over from src/tools/regression/.

Additionally, this change makes it possible to perform some other requested
changes to bsd.test.mk in an easier manner.  Coming soon.
2014-03-14 08:56:19 +00:00
jilles
6139448a86 sh: Successfully do nothing when killing a terminated job.
If a job has terminated but is still known, silently do nothing when using
the kill builtin with the job specifier. Formerly, the shell called kill()
with the process group ID that might have been reused.
2014-03-08 19:44:34 +00:00
jilles
c2f01aba00 sh: Make argstr() return where it stopped and simplify expari() using this. 2014-03-04 22:30:38 +00:00
jilles
d941f4e61c sh: Simplify expari().
Redo expari() like evalvar(). This makes the logic more understandable and
avoids possible problems if arithmetic expansion occurs if CTLESC characters
are not generated (looking backwards for CTLARI is not generally possible in
that case but the old code tried anyway).

This adds an extra argstr() recursion.
2014-03-02 22:59:34 +00:00
jilles
6891107e84 sh: Do not corrupt internal representation if LINENO inner expansion fails.
Example:
  f() { : ${LINENO+$((1/0))}; }
and call this function twice.
2014-02-27 16:54:43 +00:00
jilles
cd2fb30b68 sh: Make expari() static. 2014-02-26 21:38:42 +00:00
daichi
5fa0bd3983 sh: Add -h option to SYNOPSIS
Reviewed by:	jilles
MFC after:	soon
2014-02-25 03:05:43 +00:00
jilles
9b565c0250 sh: Allow aliases to force alias substitution on the following word.
If an alias's value ends with a space or tab, the next word is also
checked for aliases.

This is a POSIX feature. It is useful with utilities like command and
nohup (alias them to themselves followed by a space).
2014-01-26 21:19:33 +00:00
jilles
f16aa3a7ff sh: Add tests for alias names after another alias.
Since the first alias's value does not end with a blank, the next word
should not be checked for aliases.
2014-01-25 14:59:08 +00:00
jilles
55c9f20404 sh: Do not depend on parse/execute split in new alias test. 2014-01-24 23:00:35 +00:00
jilles
5b4046f60f sh: Solve the alias recursion problem in a less hackish way.
Add the space to avoid alias recursion when the alias is expanded, not when
it is added.

As a result, displaying an alias via command -v, command -V or type no
longer erroneously appends a space. Adjust the tests so they now require
this bug to be absent.
2014-01-24 16:40:51 +00:00
jilles
d93326e578 sh: Add test for nested alias. 2014-01-24 15:03:56 +00:00
jilles
296c7c9901 sh: Remove SIGWINCH handler and just check for resize before every read.
The SIGWINCH handler triggers breakage in libedit which is hard to fix; see
PR bin/169773.

Also, window size changes while a program is in foreground (and it rather
than sh will receive SIGWINCH) will now be picked up automatically.

Downside: it is now certain that a resize is only processed after pressing
<Enter>. If libedit is fixed, sh will most likely have to be changed also.

PR:		bin/180146
2014-01-14 22:56:25 +00:00
jmmv
d43a2cacc2 Replace hand-crafted Kyuafiles with automatic generation.
Redo r260506 by using the new TEST_METADATA functionality of bsd.test.mk
to mark the sh(1) and test(1) tests as not supporting root.  This is to
get rid of hand-crafted Kyuafiles for these very simple cases.

MFC after:	5 days
2014-01-14 18:45:32 +00:00
jmmv
d2680a1ea6 Run the sh(1) and test(1) tests as unprivileged.
One of the tests for test(1) fails and some of the tests for sh(1) are
silently bypassed when running as root.

To fix these tests and ensure they all run, mark the test programs for
sh(1) and test(1) as requiring an unprivileged user.  (This should and
will be the default in Kyua but isn't yet.)

MFC after:	1 week
2014-01-10 10:39:01 +00:00
jilles
98001789f4 sh(1): Discourage use of -e.
Also, do not say that ! before a pipeline is an operator, because it is
syntactically a keyword.
2014-01-03 22:56:23 +00:00
jilles
ab778292cf sh: Don't check input for non-whitespace if history is disabled.
preadbuffer() maintained a flag whether there was any non-whitespace
character. This flag is only useful when history is enabled (in that case,
lines containing only whitespace are not added to history). Instead, check
using strspn() when history is enabled.

There is an approximate 2% speedup when running
  sh -c '. /etc/rc.subr; . /etc/defaults/rc.conf; source_rc_confs'
with hot cache.
2013-12-27 15:52:18 +00:00
jilles
c039e1a7e4 sh: Simplify code related to PPID variable. 2013-12-25 16:14:02 +00:00
jilles
ffaab89202 sh: Initialize OPTIND=1 even if it came from the environment. 2013-12-24 22:38:24 +00:00
jilles
67082dae0c sh: Remove an unused variable. 2013-12-24 22:04:44 +00:00
jmmv
cfc6ad9bc6 Migrate tools/regression/bin/ tests to the new layout.
This change is a proof of concept on how to easily integrate existing
tests from the tools/regression/ hierarchy into the /usr/tests/ test
suite and on how to adapt them to the new layout for src.

To achieve these goals, this change:

- Moves tests from tools/regression/bin/<tool>/ to bin/<tool>/tests/.
- Renames the previous regress.sh files to legacy_test.sh.
- Adds Makefiles to build and install the tests and all their supporting
  data files into /usr/tests/bin/.
- Plugs the legacy_test test programs into the test suite using the new
  TAP backend for Kyua (appearing in 0.8) so that the code of the test
  programs does not have to change.
- Registers the new directories in the BSD.test.dist mtree file.

Reviewed by:	freebsd-testing
Approved by:	rpaulo (mentor)
2013-12-11 04:09:17 +00:00
jilles
b5a84f73b8 sh: Split set -x output into a separate function. 2013-12-06 22:24:37 +00:00
jilles
1d244d8c45 sh: Prefer memcpy() to strcpy() in most cases. Remove the scopy macro. 2013-11-30 21:27:11 +00:00
jilles
4e34b84587 sh: Make <&0 disable the </dev/null implicit in a background command.
Although <&0 does nothing, it is a redirection affecting standard input and
should therefore disable the </dev/null redirection implicit in a background
command.
2013-11-24 23:12:13 +00:00
jilles
367c40c277 sh: Properly quote alias output from command -v.
An alias should be printed by command -v as a command line; therefore, make
the alias definition suitable for re-input to the shell.
2013-11-10 23:00:39 +00:00
jilles
4cbf76c868 sh(1),limits(1): Document kqueues (-k) rlimit. 2013-11-01 13:57:30 +00:00
jilles
4d075e1cc9 sh: Reorder union node to reduce its size on 64-bit platforms. 2013-11-01 11:28:56 +00:00
jilles
f49e62a7eb sh: Allow trapping SIGINT/SIGQUIT after ignore because of '&'.
If job control is not enabled, background jobs started with  ... &  ignore
SIGINT and SIGQUIT so that they are not affected by such signals that are
intended for the foreground job. However, this should not prevent
reassigning a different action for these signals (as if the shell invocation
inherited these signal actions from its parent).

Austin group issue #751

Example:
  { trap - INT; exec sleep 10; } & wait
A Ctrl+C should terminate the sleep command.
2013-10-30 21:36:15 +00:00
kib
734382a525 Add a resource limit for the total number of kqueues available to the
user.  Kqueue now saves the ucred of the allocating thread, to
correctly decrement the counter on close.

Under some specific and not real-world use scenario for kqueue, it is
possible for the kqueues to consume memory proportional to the square
of the number of the filedescriptors available to the process.  Limit
allows administrator to prevent the abuse.

This is kernel-mode side of the change, with the user-mode enabling
commit following.

Reported and tested by:	pho
Discussed with:	jmg
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2013-10-21 16:46:12 +00:00
jilles
3ccfe0d544 sh: Remove one syscall when waiting for a foreground job.
The getpgrp() call is unnecessary: if there is no job control then the
result was not used at all and if there is job control then we are not a
subshell and our process group ID is equal to our process ID (rootpid).
2013-10-18 12:35:12 +00:00
sjg
292ec5d301 Updated dependencies 2013-10-13 00:24:00 +00:00
sjg
62bb106222 Merge from head 2013-09-05 20:18:59 +00:00
jilles
ad7328341a sh: Make return return from the closest function or dot script.
Formerly, return always returned from a function if it was called from a
function, even if there was a closer dot script. This was for compatibility
with the Bourne shell which only allowed returning from functions.

Other modern shells and POSIX return from the function or the dot script,
whichever is closest.

Git 1.8.4's rebase --continue depends on the POSIX behaviour.

Reported by:	Christoph Mallon, avg
2013-09-04 22:10:16 +00:00
jilles
ae84eb37bd sh: Fix race condition with signals and wait or set -T.
The change in r238888 was incomplete. It was still possible for a trapped
signal to arrive before the shell went to sleep (sigsuspend()) because a
check was missing or because the signal arrived before in_waitcmd was set.

On SMP, this bug sometimes caused the builtins/wait4.0 test to take 1 second
to execute; it then might or might not fail. On UP, the test almost always
failed.
2013-09-02 21:57:46 +00:00
jilles
6d50a40d08 sh: Simplify list() in the parser.
The erflag argument was only used by old-style (``) command substitutions.
We can remove it and handle the special case in the command substitution
code.
2013-08-30 20:50:28 +00:00
jilles
fc9c61263d sh: Separate out nbinary allocation into a function. 2013-08-30 20:37:52 +00:00
jilles
67c1af2857 sh: Use makename() where possible. 2013-08-30 20:13:33 +00:00
jilles
e60f4ea26b sh: Add a function for the case where one token is required in the parse. 2013-08-30 13:25:15 +00:00
jilles
8b7d6031e6 sh: Recognize "--" as end of options in type builtin.
This implementation makes minimal changes: command names starting with "-"
(other than "--") can still be queried normally.
2013-08-30 12:09:59 +00:00
jilles
415d59b5d8 sh: Cast -1 to pointer rather than pointer to variable of wrong type.
NEOF needs to be a non-null pointer distinct from valid union node pointers.
It is not dereferenced.

The new NEOF is much like SIG_ERR except that it is an object pointer
instead of a function pointer.

The variable tokpushback can now be static.
2013-08-30 10:45:02 +00:00
jilles
59cd1d2d27 sh: Recognize "--" as end of options in alias builtin.
Aliases starting with "-" (which are non-POSIX) will need to be preceded by
an alias not starting with "-" or the newly added "--".
2013-08-25 11:42:53 +00:00
jilles
e694117f82 sh: Disallow empty simple commands.
As per POSIX, a simple command must have at least one redirection,
assignment word or command word.

These occured in rare cases such as  eval "f()" .

The extension of allowing no commands inside { }, if, while, for, etc.
remains.
2013-08-25 10:57:48 +00:00
jilles
f53205fcfa sh: Reject ++ and -- in arithmetic.
POSIX does not require ++ and -- in arithmetic. It is probably more useful
to reject them than to treat ++x and --x as x silently.

Note that the behaviour of increment and decrement can be obtained via
(x+=1), ((x+=1)-1), (x-=1) and ((x-=1)+1).

PR:		bin/176444
2013-08-24 20:06:00 +00:00
jilles
739c3b84b8 sh: Do not prematurely discard stopped jobs in a wait builtin.
If a job is specified to 'wait', wait for it to complete. Formerly, in
interactive mode, the job was deleted if it stopped.

If no jobs are specified in interactive mode, 'wait' still waits for all jobs
to complete or stop.

In non-interactive mode, WUNTRACED is not passed to wait3() so stopped jobs
are not detected.

PR:		bin/181435
2013-08-24 09:57:32 +00:00
jilles
e5ea815310 sh: Remove unnecessary reset functions.
These are already handled by exception handlers.
2013-08-16 20:24:41 +00:00
jilles
c433419763 sh: Recognize "--" as end of options in bg/fg/jobid builtins. 2013-08-16 13:56:43 +00:00
jilles
c71191e0d9 sh: Recognize "--" as end of options in local builtin. 2013-08-14 21:59:48 +00:00
jilles
79360f25fc sh: Allow a lone redirection before '|', ';;' or ';&'.
Example: </dev/null | :

PR:		181240
MFC after:	1 week
2013-08-14 19:34:13 +00:00
jilles
44afceea6d sh: Remove an incorrect comment. 2013-07-25 20:50:35 +00:00
jilles
45d56acf59 sh: Remove #define MKINIT.
MKINIT only served for the removed mkinit. Many variables can be static now.
2013-07-25 19:48:15 +00:00
jilles
0ad2a46f33 sh: Remove mkinit.
Replace the RESET blocks with regular functions and a reset() function that
calls them all.

This code generation tool is unusual and does not appear to provide much
benefit. I do not think isolating the knowledge about which modules need to
be reset is worth an almost 500-line build tool and wider scope for
variables used by the reset functions.

Also, relying on reset functions is often wrong: the cleanup should be done
in exception handlers so that no stale state remains after 'command eval'
and the like.
2013-07-25 15:08:41 +00:00
jilles
34bfb313cd sh: Remove output.c's reset() handler.
These cleanup operations are not needed because they are already performed
after an optimized command substitution (whether there was an error or not).
2013-07-25 13:09:17 +00:00
jilles
e87e084ab7 sh: Do not read from stdin if an error occurs during -i -c cmd.
Although using -i with -c does not seem very useful, it seems inappropriate
to read commands from the terminal in this case.

Side effect: if the -s -c extension is used and the -s option is turned off
using 'set +s' during the interactive part, the shell now exits after an
error or interrupt. Note that POSIX only specifies -s as option to sh, not
to set.

See also Austin Group issue #718.
2013-07-12 15:29:41 +00:00
jilles
836c13641c sh: Do not close(-1) if pipe() fails. 2013-06-28 21:47:08 +00:00
jilles
ceb4b0c61b sh(1): A subshell environment has its own rlimits (ulimit).
This has always been the case and is intended (just like cd).

This matches Austin group issue #706.
2013-06-14 22:06:18 +00:00
jilles
7deb35147a sh(1): Document new features in wait builtin.
PR:		176916
2013-06-05 19:54:28 +00:00
jilles
34ac6a12ea sh: Return status 127 for unknown jobs in wait builtin.
This is required by POSIX, at least for pids that are not known child
processes.

Other problems with job specifications still cause wait to abort with
exit status 2.

PR:		176916
2013-06-05 19:40:52 +00:00
jilles
dbbdb077d8 sh: Allow multiple operands in wait builtin.
This is only part of the PR; the behaviour for unknown/invalid pids/jobs
remains unchanged (aborts the builtin with status 2).

PR:		176916
Submitted by:	Vadim Goncharov
2013-06-05 19:08:22 +00:00
jilles
e0dffd814f sh: Remove linked list of stack marks.
The linked list of stack marks may cause problems if the allocation stack is
used between an exception and a higher-level popstackmark(), as it may then
touch a stack mark that is local to a function which has returned.

Also, the adjustment compares to a pointer passed to realloc(), which is
undefined behaviour.

Instead of adjusting stack marks when reallocating stack blocks, ensure that
such an adjustment is never necessary by fixing a small piece of memory in
place at a stack mark. This also simplifies the code.

To avoid the problems reported in bin/175922, it remains necessary to call
setstackmark() after popstackmark() if the stack mark remains in use.
2013-05-11 20:51:00 +00:00
eadler
69c7d77aef Fix two typos
Reviewed by:	jilles
2013-05-09 21:29:16 +00:00
jilles
135786413b sh: Use O_CLOEXEC and F_DUPFD_CLOEXEC instead of separate fcntl() call. 2013-05-05 10:51:40 +00:00
jilles
3f27c69169 sh: Improve error handling in read builtin:
* If read -t times out, return status as if interrupted by SIGALRM
  (formerly 1).
* If a trapped signal interrupts read, return status 128+sig (formerly 1).
* If [EINTR] occurs but there is no trap, retry the read (for example
  because of a SIGWINCH in interactive mode).
* If a read error occurs, write an error message and return status 2.

As before, a variable assignment error returns 2 and discards the remaining
data read.
2013-05-03 15:28:31 +00:00
jilles
9081eaa709 sh: Don't consider jobs -s/-p as reporting the status of jobs.
This ensures that something like j=$(jobs -p) does not prevent any
status from being written to the terminal.
2013-04-27 14:07:25 +00:00
joel
1b87173706 Document a few expansions for the $PS1 and $PS2 environmental variables.
PR:		173410
Submitted by:	Derek Wood <ddwood@outlook.com>
Reviewed by:	jilles
2013-04-21 19:55:38 +00:00
sjg
97d8b94956 sync from head 2013-04-12 20:48:55 +00:00
jilles
8b20292658 sh: Don't modify exit status when break/continue/return passes !.
This matches what would happen if  ! P  were to be replaced with
if P; then false; else true; fi.

Example:
  f() { ! return 0; }; f
2013-04-12 15:19:35 +00:00
jilles
6990a1cb76 sh: Add const to nodesavestr(). 2013-04-07 16:28:36 +00:00
jilles
7236eb0dda sh: Write as much into the heredoc pipe as possible, to avoid forking.
Use non-blocking I/O to write as much as the pipe will accept (often 64K,
but it can be as little as 4K), avoiding the need for the ugly PIPESIZE
constant. If PIPESIZE was set too high, a deadlock would occur.
2013-04-02 21:34:38 +00:00
jilles
3deb97fc0b sh: Fix various compiler warnings.
It now passes WARNS=7 with clang on i386.

GCC 4.2.1 does not understand setjmp() properly so will always trigger
-Wuninitialized. I will not add the volatile keywords to suppress this.
2013-04-01 17:18:22 +00:00
joel
2f6a71b3ae Minor mdoc fix. 2013-03-29 08:12:09 +00:00
jilles
e740fd1ca7 sh(1): Mention possible ambiguities with $(( and ((.
In some other shells, things like $((a);(b)) are command substitutions.

Also, there are shells that have an extension ((ARITH)) that evaluates an
arithmetic expression and returns status 1 if the result is zero, 0
otherwise. This extension may lead to ambiguity with two subshells starting
in sequence.
2013-03-24 22:48:45 +00:00
jilles
d967c25f5c sh: Recognize "--" and explicitly reject options in wait builtin.
If syntactically invalid job identifiers are to be taken as jobs that exited
with status 127, this should not apply to options, so that we can add
options later if need be.
2013-03-15 20:29:31 +00:00
sjg
6d37b86f2b Updated dependencies 2013-03-11 17:21:52 +00:00
jilles
d0155c5dcc sh: When executing a trap, keep exit status along with evalskip.
This ensures 'return' in a trap returns the correct status to the caller.

If evalskip is not set or if it is overridden by a previous evalskip, keep
the old behaviour of restoring the exit status from before the trap.
2013-03-03 17:33:59 +00:00
jilles
6d26f3b024 sh: If a SIGINT or SIGQUIT interrupts "wait", return status 128+sig. 2013-02-23 22:50:57 +00:00
jilles
d08a4b37f1 sh: Fix a crash with the stackmark code.
If a stack mark is set while the current stack block is empty, the stack
block may move later on (because of realloc()) and the stack mark needs to
be updated. This updating does not happen after popstackmark() has been
called; therefore, call setstackmark() again if the stack mark is still
being used.

For some reason, this only affects a few users. I cannot reproduce it. The
situation seems quite rare as well because an empty stack block would
usually be freed (by popstackmark()) before execution reaches a
setstackmark() call.

PR:		175922
Tested by:	KT Sin
2013-02-19 23:46:51 +00:00
sjg
0ee5295509 Updated dependencies 2013-02-16 01:23:54 +00:00
obrien
3028e3f8ab Sync with HEAD. 2013-02-08 16:10:16 +00:00
jilles
89d287b040 sh: Simplify mksyntax and make it fit for cross-compiling.
Now it outputs fixed files, which use constants provided by the C standard
library to determine appropriate values for the target machine.

Before, mksyntax inspected the host machine which resulted in subtle
breakage if e.g. char is signed on the host and unsigned on the target such
as when cross-compiling on x86 for ARM.

Tested using -funsigned-char on amd64. Compiling build-tools without it and
sh itself with it causes various tests to fail without this change but not
with this change. With consistent -funsigned-char, tests pass with or
without this change.

The mksyntax program could be removed and syntax.c and syntax.h committed to
the repository.

Submitted by:	Christoph Mallon
MFC after:	2 weeks
2013-02-07 22:42:33 +00:00
jilles
bb422352b5 sh: Fix a comment. 2013-02-07 21:24:10 +00:00
delphij
c27302a50c Catch TRACE parameters up with r238888. This change is only needed when
debugging is enabled.
2013-02-07 19:00:54 +00:00
jilles
f923241b93 sh: Do not test for digit_contig in mksyntax.
ISO/IEC 9899:1999 (E) 5.2.1p3 guarantees that the values of the characters
0123456789 are contiguous.

The generated syntax.c and syntax.h remain the same.

Submitted by:	Christoph Mallon
2013-02-05 22:54:09 +00:00
jilles
6fa139d56e sh: Expand here documents in the current process.
Expand here documents at the same point other redirections are expanded but
use a non-fork subshell environment (like simple command substitutions) for
compatibility. Substitition errors result in an empty here document like
before.

As a result, a fork is avoided for short (<4K) expanded here documents.

Unexpanded here documents (with quoted end marker after <<) are not affected
by this change. They already only forked when >4K.

Side effects:
* Order of expansion is slightly different.
* Slow expansions are not executed in parallel with the redirected command.
* A non-fork subshell environment is subtly different from a forked process.
2013-02-03 15:54:57 +00:00
jilles
43d0e15091 sh: Prefer our character classification functions to <ctype.h>. 2013-01-31 22:10:57 +00:00
jilles
db77635860 sh: Show negated commands (!) in jobs output. 2013-01-31 21:50:44 +00:00
joel
30cb6e2951 Add FILES section.
Discussed with:	jilles
2013-01-22 18:02:58 +00:00
joel
16ff7601a6 Change the $ENV example to use .shrc instead of .shinit. This is consistent
with what we use in /usr/share/skel/dot.profile.

Discussed with: jilles
2013-01-20 22:25:25 +00:00
jilles
81fe037601 sh: Move some stackmarks to fix high memory usage in some loops.
If a loop contained certain commands (such as redirected compound commands),
the temporary memory for the redirection was not freed between iterations of
the loop but only after the loop.

Put a stackmark in evaltree(), freeing memory whenever a node has been
evaluated. Some other stackmarks are then redundant; remove them.

Example:
  while :; do { :; } </dev/null; done
2013-01-20 21:28:05 +00:00
jilles
cbc4c398d4 sh: Remove mkinit's initialization routine.
Instead, call the only init function left directly from main().
2013-01-20 12:44:50 +00:00
jilles
3d71dcf315 sh: Replace an mkinit use with an initialization. 2013-01-19 22:12:08 +00:00
jilles
705fd8516a sh: Simplify cd-hash interaction.
Instead of rechecking relative paths for all hashed utilities after a cd,
track if any utility in cmdtable depends on a relative path in PATH.
If there is such a utility, cd clears the entire table.

As a result, the '*' in hash no longer happens.
2013-01-14 16:40:50 +00:00
jilles
d139340997 sh: Pass $? to command substitution containing compound/multiple commands.
Example:
  false; echo $(echo $?; :)
2013-01-14 12:20:55 +00:00
jilles
b479a582c3 sh: Fix crash when parsing '{ } &'.
MFC after:	1 week
2013-01-13 19:26:33 +00:00
jilles
1af5c5cc5f sh: Don't lose $? when backquoted command ends with semicolon or newline.
An empty simple command was added and overwrote the exit status with 0.

This affects `...` but not $(...).

Example:
  v=`false;`; echo $?
2013-01-13 19:19:40 +00:00
jilles
da771ef7de sh: Prefer strsignal() to accessing sys_siglist directly.
Accessing sys_siglist directly requires rtld to copy it from libc to the sh
executable's BSS. Also, strsignal() will put in the signal number for
unknown signals (FreeBSD-specific) so we need not do that ourselves.

Unfortunately, there is no function for sys_signame.
2012-12-25 14:17:09 +00:00
jilles
e5dd01ef24 sh: Detect and flag write errors on stdout in builtins.
If there is a write error on stdout, a message will be printed (to stderr)
and the exit status will be changed to 2 if it would have been 0 or 1.

PR:		bin/158206
2012-12-12 22:01:10 +00:00
jilles
bba3ac5a32 sh: Remove an unused variable. 2012-11-14 14:08:24 +00:00
jilles
be4aea389c sh: Forward-declare struct alias instead of giving up type safety via void * 2012-11-11 15:13:24 +00:00
jilles
9009053e85 sh: Fix two issues when an alias is redefined:
* The last character is not displayed.
 * If the alias ends with itself (as a word), an infinite memory-eating loop
   occurs.

If an alias is defined initially, a space is appended to avoid recursion but
this did not happen when an alias was later modified.

PR:		bin/173418
Submitted by:	Daniel F.
MFC after:	1 week
2012-11-08 13:33:48 +00:00
jilles
aef0db8f9e sh: Change cmdtype in tblentry from short to signed char.
If this is a smaller type than int anyway, we can make it the smallest
possible.
2012-11-05 17:52:18 +00:00
sjg
778e93c51a Sync from head 2012-11-04 02:52:03 +00:00
jilles
abfb4133f0 sh: Use C99 flexible array instead of accessing array beyond bounds.
Although sufficient memory is available for a longer string in cmdname,
this is undefined behaviour anyway.

Side effect: for alignment reasons, an additional byte of memory is
allocated per hashed command.
2012-11-03 22:23:08 +00:00
joel
72961fb7f5 mdoc: don't nest displays. The markup here isn't adding anything anyway.
Fixes a mandoc lint warning.

Discussed with:	brueffer, Jason McIntyre <jmc@kerhand.co.uk>
2012-10-14 13:59:17 +00:00
jilles
99ca87dd2d sh: Prefer internal nextopt() to libc getopt().
This reduces code duplication and code size.

/usr/bin/printf is not affected.

Side effect: different error messages when certain builtins are passed
invalid options.
2012-09-15 21:56:30 +00:00
jilles
126621e68f sh: Reduce code duplication: use setinputfile instead of open/setinputfd
combination.

MFC after:	2 weeks
2012-09-14 15:22:15 +00:00
joel
2c9fc028f5 Minor mdoc fix. 2012-09-11 17:57:03 +00:00
jilles
88d1219324 sh: Remove XXX comment about removing nextopt().
Using nextopt() avoids depending on the BSD-specific optreset feature in
getopt() and reduces code size (both source and binary).
2012-09-08 19:24:03 +00:00
marcel
9dd41e3647 Sync FreeBSD's bmake branch with Juniper's internal bmake branch.
Requested by: Simon Gerraty <sjg@juniper.net>
2012-08-22 19:25:57 +00:00
jilles
0df7adbcbe sh: Fix EINTR race condition in "wait" and "set -T" using sigsuspend().
When waiting for child processes using "wait" or if "set -T" is in effect, a
signal interrupts the wait. Make sure there is no window where the signal
handler may be invoked (setting a flag) just before going to sleep.

There is a similar race condition in the shell language, but scripts can
avoid it by exiting from the trap handler or enforcing synchronization using
a fifo.

If SIGCHLD is not trapped, a signal handler must be installed for it. Only
install this handler for the duration of the wait to avoid triggering
unexpected [EINTR] errors elsewhere.

Note that for some reason only SIGINT and SIGQUIT interrupt a "wait"
command. This remains the case.
2012-07-29 18:04:38 +00:00
jilles
68e92f2ba9 sh: Do not ask for stopped/continued processes if we do not need them
rather than retrying wait3 if they happen.
2012-07-28 15:13:48 +00:00
jilles
acf68e5eb4 sh: Inline waitproc() into its only caller. 2012-07-28 14:56:50 +00:00
jilles
d73fbcf117 sh: Track continued jobs (even if not continued by bg or fg).
This uses wait3's WCONTINUED flag.

There is no message for this. The change is visible in "jobs" or if the job
stops again.
2012-07-28 14:32:55 +00:00
jilles
9ceedcafff sh: Reset pendingsigs before checking pending traps, not after.
Otherwise, a signal arriving at exactly the right moment might not be
processed until another signal arrived.
2012-07-15 11:18:52 +00:00
jilles
4d945ad412 sh: Remove unused variable in_dowait. 2012-07-15 10:49:16 +00:00
jilles
689774f8e7 sh: Expand assignment-like words specially for export/readonly/local.
Examples:
  export x=~
now expands the tilde
  local y=$1
is now safe, even if $1 contains IFS characters or metacharacters.

For a word to "look like an assignment", it must start with a name followed
by an equals sign, none of which may be quoted.

The special treatment applies when the first word (potentially after
"command") is "export", "readonly" or "local". There may be quoting
characters but no expansions. If "local" is overridden with a function there
is no special treatment ("export" and "readonly" cannot be overridden with a
function).

If things like
  local arr=(1 2 3)
are ever allowed in the future, they cannot call a "local" function. This
would either be a run-time error or it would call the builtin.

This matches Austin Group bug #351, planned for the next issue of POSIX.1.

PR:		bin/166771
2012-07-15 10:19:43 +00:00
pfg
676e6cd9af Merge libedit adjustment from NetBSD.
On recent versions of NetBSD's libedit, el_gets
now sets el_len to -1 on error so we can
distinguish between a NULL string and an error.

This fixes sh from exiting with newer versions
of libedit now allowing EINTR to return.

Obtained from:	NetBSD
Reviewed by:	jilles
MFC after:	3 weeks
2012-07-11 22:17:58 +00:00
marcel
e7223eeffb Work better with how make/bmake works:
1.  Avoid a cd back into ${.CURDIR} to run mkbuiltins when we know make
    will first cd into ${.OBJDIR}. Keep the cwd to what make sets it to.
2.  Don't tell mkbuiltins where to write to (= ${.OBJDIR}), but where to
    get sources from (= ${.CURDIR}). This to compensate for point 1.

This fixes a problem with bmake's mk files that optimize ${.OBJDIR} to
expand to "." after changing cwd, not taking into account that the
target is pretty much undoing that and not getting the full path to the
object tree anymore.
2012-05-24 19:48:15 +00:00
jilles
5d003d3cdb sh: Remove an unused header.
The read builtin no longer does things with termios.
2012-05-15 22:50:47 +00:00
joel
82a81c78d9 Minor mdoc nits. 2012-05-13 14:16:04 +00:00
joel
0ef023ae5b mdoc: fix column names, indentation, column separation within each row, and
quotation. Also make sure we have the same amount of columns in each row as
the number of columns we specify in the head arguments.

Reviewed by:	brueffer
2012-04-07 09:05:30 +00:00
jilles
17b84cf238 sh: Fix build with -DDEBUG=2.
Reported by:	Kristof Provost
MFC after:	1 week
2012-04-02 17:16:24 +00:00
eadler
1ef5fe44d3 Remove trailing whitespace per mdoc lint warning
Disussed with:	gavin
No objection from:	doc
Approved by:	joel
MFC after:	3 days
2012-03-29 05:02:12 +00:00
joel
2f38aba47a Remove superfluous paragraph macro. 2012-03-25 09:20:14 +00:00
jilles
0458d57ea3 sh: Make 'hash' return 1 if at least one utility is not found.
Reported by:	lme
2012-02-11 21:06:45 +00:00
jilles
44943dcf5b sh: Fix swapped INTON/INTOFF.
A possible consequence of this bug was a memory leak if SIGINT arrived
during a 'set' command (listing variables).

MFC after:	1 week
2012-02-04 23:29:07 +00:00
jilles
c9a60ad55a sh: Use vfork in a few common cases.
This uses vfork() for simple commands and command substitutions containing a
single simple command, invoking an external program under certain conditions
(no redirections or variable assignments, non-interactive shell, no job
control). These restrictions limit the amount of code executed in a vforked
child.

There is a large speedup (for example 35%) in microbenchmarks. The
difference in buildkernel is smaller (for example 0.5%) but still
statistically significant. See
http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/037581.html
for some numbers.

The use of vfork() can be disabled by setting a variable named
SH_DISABLE_VFORK.
2012-02-04 23:12:14 +00:00
charnier
11c560ec5e Add prototypes, ANSIfy functions definitions to reduce WARNS=6 output. 2012-01-25 08:42:19 +00:00
jilles
ae59680813 sh: Fix $? in the first command of a 'for'.
In the first command of a 'for', $? should be the exit status of the last
pipeline (command substitution in the word list or command before 'for'),
not always 0.
2012-01-22 14:00:33 +00:00
jilles
5ee1538f37 sh: Remove "kill" example function, which is superseded by the kill builtin
MFC after:	1 week
2012-01-21 20:39:17 +00:00
dumbbell
8b387a2d35 sh: Fix execution of multiple statements in a trap when evalskip is set
Before this fix, only the first statement of the trap was executed if
evalskip was set. This is for example the case when:
    o  "-e" is set for this shell
    o  a trap is set on EXIT
    o  a function returns 1 and causes the script to abort

Reviewed by:	jilles
MFC after:	2 weeks
2012-01-16 11:07:46 +00:00
jilles
0ae130814a sh: Fix some bugs with exit status from case containing ;&.
Also, rework evalcase() to not evaluate any tree. Instead, return the
NCLISTFALLTHRU node and handle it in evaltree().

Fixed bugs:

* If a ;& list with non-zero exit status is followed by an empty ;; or final
  list, the exit status of the case command should be equal to the exit
  status of the ;& list, not 0.

* An empty ;& case should not reset $?.
2012-01-15 21:39:38 +00:00
jilles
6253417a70 sh: Fix two bugs with case and exit status:
* If no pattern is matched, POSIX says the exit status shall be 0 (even if
  there are command substitutions).
* If a pattern is matched and there are no command substitutions, the first
  command should see the $? from before the case command, not always 0.
2012-01-15 20:04:05 +00:00
jilles
b166a3a409 sh: Change input buffer size from 1023 to 1024.
PR:		bin/161756
2012-01-14 22:46:18 +00:00
jilles
51c9a6e5fa sh: Fix out of bounds array access when trap is used with an invalid signal.
MFC after:	1 week
2012-01-14 21:54:12 +00:00
jilles
69e6f0d416 sh: Properly show "Not a directory" error in cd builtin.
The errno message display added in r222292 did not take attempting to
cd to a non-directory or something that cannot be stat()ed into account.

PR:		bin/164070
MFC after:	10 days
2012-01-13 23:32:27 +00:00
jilles
8152f4c192 sh: Make various functions static. 2012-01-01 22:17:12 +00:00
jilles
bbd60abdf3 sh: Remove unused function scopyn(). 2012-01-01 22:15:38 +00:00
jilles
109579dd42 sh: Make patmatch() non-recursive. 2012-01-01 20:50:19 +00:00
jilles
c935aad28d sh: Allow quoting ^ and ] in bracket expressions. 2011-12-28 23:51:17 +00:00
jilles
0121cc11e8 sh: Use dirent.d_type in pathname generation.
This improves performance for globs where a slash or another component
follows a component with metacharacters by eliminating unnecessary attempts
to open directories that are not.
2011-12-28 23:40:46 +00:00
jilles
f922478439 sh: Cache de->d_namlen in a local variable. 2011-12-28 23:30:17 +00:00
jilles
45b77afcc1 sh: Do not force special builtins non-special in optimized command subst.
This is not necessary: errors are already caught in evalbackcmd() and
forcelocal handles changes to variables.

Note that this depends on r223024.

MFC after:	4 weeks
2011-12-28 22:10:12 +00:00
jilles
519601e533 sh: Remove impossible evalskip check in 'for'. 2011-11-27 00:09:59 +00:00
jilles
49f86abcab sh: Reduce one level of evaltree() recursion when executing 'case'.
Free expanded case text before executing commands.
Remove impossible evalskip checks (expanding an argument cannot set
evalskip anymore since $(break) and the like are properly executed in a
subshell environment).
2011-11-26 23:28:31 +00:00
jilles
1018bfcafa sh: Allow unsetting OPTIND.
Note that only assigning the decimal value 1 resets getopts, as before.
2011-11-20 21:48:50 +00:00
jilles
91e70324c8 sh: Remove undefined behaviour due to overflow in +/-/* in arithmetic.
With i386 base gcc and i386 base clang, arith_yacc.o remains unchanged.
2011-11-08 23:54:39 +00:00
jilles
96a8647851 sh(1): Improve documentation of field splitting. 2011-11-05 21:56:45 +00:00
jilles
845b2cab39 sh(1): Extend documentation about subshells.
Because sh executes commands in subshell environments without forking in
more and more cases (particularly from 8.0 on), it makes sense to describe
subshell environments more precisely using ideas from POSIX, together with
some FreeBSD-specific items.

In particular, the hash and times builtins may not behave as if their state
is copied for a subshell environment while leaving the parent shell
environment unchanged.
2011-07-10 15:02:25 +00:00
jilles
75acd605ce sh: Include <limits.h> instead of non-standard <sys/limits.h>. 2011-06-26 20:12:05 +00:00
jilles
76e2eec6e1 sh(1): Improve documentation of shell patterns:
* Shell patterns are also for ${var#pat} and the like.
* An '!' by itself will not trigger pathname generation so do not call it a
  meta-character, even though it has a special meaning directly after an
  '['.
* Character ranges are locale-dependent.
* A '^' will complement a character class like '!' but is non-standard.

MFC after:	1 week
2011-06-24 22:08:26 +00:00
jilles
f5e9daf260 sh(1): Document the case command better.
Suggested by:	netchild
Reviewed by:	gjb
2011-06-24 20:23:50 +00:00
kevlo
c1214139ee Remove duplicated header files 2011-06-24 07:29:04 +00:00
jilles
f78ebc168b sh: Remove special support for background simple commands.
It expands the arguments in the parent shell process, which is incorrect.
2011-06-18 23:58:59 +00:00
jilles
a380f591f1 sh: Add do-nothing -h option.
POSIX requires a -h option to sh and set, to locate and remember utilities
invoked by functions as they are defined. Given that this
locate-and-remember process is optional elsewhere, it seems safe enough to
make this option do nothing.

POSIX does not specify a long name for this option. Follow ksh in calling it
"trackall".
2011-06-18 23:43:28 +00:00
jilles
c7a72567a8 sh: Add case statement fallthrough (with ';&' instead of ';;').
Replacing ;; with the new control operator ;& will cause the next list to be
executed as well without checking its pattern, continuing until a list ends
with ;; or until the end of the case statement. This is like omitting
"break" in a C "switch" statement.

The sequence ;& was formerly invalid.

This feature is proposed for the next POSIX issue in Austin Group issue
#449.
2011-06-17 13:03:49 +00:00
jilles
f84ee4a43b sh: Skip variables with invalid names in "set", "export -p", "readonly -p".
This ensures the output of these commands is valid shell input.
2011-06-17 10:21:24 +00:00
jilles
00d33feb9d sh: Reduce unnecessary forks with eval.
The eval special builtin now runs the code with EV_EXIT if it was run
with EV_EXIT itself.

In particular, this eliminates one fork when a command substitution contains
an eval command that ends with an external program or a subshell.

This is similar to what r220978 did for functions.
2011-06-16 21:50:28 +00:00
jilles
84b55be725 sh: Add support for named character classes in bracket expressions.
Example:
  case x in [[:alpha:]]) echo yes ;; esac
2011-06-15 21:48:10 +00:00
jilles
16addb8673 sh: Fix duplicate prototypes for builtins.
Have mkbuiltins write the prototypes for the *cmd functions to builtins.h
instead of builtins.c and include builtins.h in more .c files instead of
duplicating prototypes for *cmd functions in other headers.
2011-06-13 21:03:27 +00:00
jilles
91789615b4 sh: Save/restore changed variables in optimized command substitution.
In optimized command substitution, save and restore any variables changed by
expansions (${var=value} and $((var=assigned))), instead of trying to
determine if an expansion may cause such changes.

If $! is referenced in optimized command substitution, do not cause jobs to
be remembered longer.

This fixes $(jobs $!) again, simplifies the man page and shortens the code.
2011-06-12 23:06:04 +00:00
jilles
59713c6c29 sh: Fix locale-dependent ranges in bracket expressions.
When I added UTF-8 support in r221646, the LC_COLLATE-based ordering broke
because of sign extension of char.

Because of libc restrictions, this does not work for UTF-8. For UTF-8
locales, ranges always use character code order.
2011-06-12 12:54:52 +00:00
jilles
01ffb61bd7 sh: Read .profile from the home directory (or / if HOME is not set).
In most cases, login shells are started from the home directory, but not in
all, such as xterm -ls.

This commit depends on r222957 for read_profile() performing parameter
expansion.

PR:		bin/50569
2011-06-12 10:13:48 +00:00
jilles
742a97ee0a sh: Do parameter expansion on ENV before using it.
This is required by POSIX, and allows things like ENV=\$HOME/.shrc.

Note that tilde expansion is explicitly not performed.
2011-06-10 22:42:00 +00:00
jilles
bd55770c94 sh: Do parameter expansion before printing PS4 (set -x).
The function name expandstr() and the general idea of doing this kind of
expansion by treating the text as a here document without end marker is from
dash.

All variants of parameter expansion and arithmetic expansion also work (the
latter is not required by POSIX but it does not take extra code and many
other shells also allow it).

Command substitution is prevented because I think it causes too much code to
be re-entered (for example creating an unbounded recursion of trace lines).

Unfortunately, our LINENO is somewhat crude, otherwise PS4='$LINENO+ ' would
be quite useful.
2011-06-09 23:12:23 +00:00
jilles
fba76e8544 sh: Fix $? in heredocs on simple commands.
PR:		bin/41410
2011-06-05 14:13:15 +00:00
jilles
b6c58c05a4 sh: Improve error message if the script cannot be opened.
Avoid "<nosuchfile>: cannot open <nosuchfile>: ...".
2011-06-04 22:19:00 +00:00
jilles
e193bb76ce sh: Reduce more needless differences between error messages. 2011-06-04 15:05:52 +00:00
jilles
0383d0751d sh: Honour -n while processing -c string. 2011-06-04 11:28:42 +00:00
jilles
aa5140b7ec sh: Remove the "exp" builtin.
The "exp" builtin is undocumented, non-standard and not very useful.

If exp's return value is not used, something like
VAR=$(exp EXPRESSION)
is equivalent to
VAR=$((EXPRESSION))
except that errors in the expression are fatal and quoting special
characters is not needed in the latter case.

If exp's return value is used, something like
if exp EXPRESSION >/dev/null
can be replaced by
if [ $((EXPRESSION)) -ne 0 ]
with similar differences.

The exp-run showed that "let" is close enough to bash's and ksh's builtin
that removing it would break a few ports. Therefore, "let" remains in 9.x.

PR:		bin/104432
Exp-run done by: pav (with some other sh(1) changes)
2011-05-27 20:53:07 +00:00
jilles
5c7c156d1e sh: Correct criterion for using CDPATH in cd.
CDPATH should be ignored not only for pathnames starting with '/' but also
for pathnames whose first component is '.' or '..'.

The man page already describes this behaviour.
2011-05-27 20:01:46 +00:00
jilles
c4d8d18073 sh: Various updates to the TOUR document. 2011-05-27 16:00:37 +00:00
jilles
543f63b8dc sh: Fix unquoted $@/$* if IFS=''.
If IFS is null, unquoted $@/$* should still expand to separate words.
This differs from quoted $@ (which does not depend on IFS) in that pathname
generation is performed and empty words are removed.
2011-05-27 15:56:13 +00:00
jilles
b5e2a8cd97 sh: Show errno messages in cd. 2011-05-25 21:38:16 +00:00
jilles
f22e5fb539 sh: Remove obsolete token type TENDBQUOTE.
This token type was related to Almquist's original version of backquotes
that could not nest and fell into disuse fairly soon.
2011-05-22 15:24:56 +00:00
uqs
bfe0a2ea26 Fix some typos under bin/
Found by:	codespell
2011-05-22 14:03:46 +00:00
jilles
b78a69b708 sh: Fix bss-based buffer overflow in . builtin.
If the length of a directory in PATH together with the given filename
exceeded FILENAME_MAX (which may happen even for pathnames that work), a
static buffer was overflown.

The static buffer is unnecessary, we can use the stalloc() stack.

Obtained from:	NetBSD
MFC after:	1 week
2011-05-22 12:12:28 +00:00
jilles
3dd8ae4222 sh: Expand aliases after assignments and redirections. 2011-05-21 22:03:06 +00:00
jilles
8c45da6d46 sh: Implement the cd -e flag proposed for the next POSIX issue.
This reflects failure to determine the pathname of the new directory in the
exit status (1). Normally, cd returns successfully if it did chdir() and the
call was successful.

In POSIX, -e only has meaning with -P; because our -L is not entirely
compliant and may fall back to -P mode, -e has some effect with -L as well.
2011-05-20 22:55:18 +00:00
jilles
343e29c626 sh: Allow terminating a heredoc with a terminator at EOF without a newline.
This is sometimes used with eval or old-style command substitution, and most
shells other than ash derivatives allow it.

It can also be used with scripts that violate POSIX's requirement on the
application that they end in a newline (scripts must be text files except
that line length is unlimited).

Example:
v=`cat <<EOF
foo
EOF`
echo $v

This commit does not add support for the similar construct with new-style
command substitution, like
v=$(cat <<EOF
foo
EOF)
This continues to require a newline after the terminator.
2011-05-20 16:03:36 +00:00
jilles
14adb603ef sh: Minor optimization to output from ulimit/export/readonly.
No functional change is intended.
2011-05-15 22:09:27 +00:00
jilles
ee209e5ce8 sh: Avoid close(-1) when evaluating a multi-command pipeline.
Valgrind complains about this.
2011-05-15 17:00:43 +00:00
jilles
c9be2081e0 sh: Add \u/\U support (in $'...') for UTF-8.
Because we have no iconv in base, support for other charsets is not
possible.

Note that \u/\U are processed using the locale that was active when the
shell started. This is necessary to avoid behaviour that depends on the
parse/execute split (for example when placing braces around an entire
script). Therefore, UTF-8 encoding is implemented manually.
2011-05-08 17:40:10 +00:00
jilles
bf0e27838c sh: Optimize variable code by storing the length of the name.
Obtained from:	NetBSD
2011-05-08 16:15:50 +00:00
jilles
b838671bb4 sh(1): Update BUGS section for UTF-8 support. 2011-05-08 14:03:44 +00:00
jilles
8ac39aa5be sh: Add UTF-8 support to pattern matching.
?, [...] patterns match codepoints instead of bytes. They do not match
invalid sequences. [...] patterns must not contain invalid sequences
otherwise they will not match anything. This is so that ${var#?} removes the
first codepoint, not the first byte, without putting UTF-8 knowledge into
the ${var#pattern} code. However, * continues to match any string and an
invalid sequence matches an identical invalid sequence. (This differs from
fnmatch(3).)
2011-05-08 11:32:20 +00:00
jilles
8bbce85526 sh: Add UTF-8 support to ${#var}.
If the current locale uses UTF-8, ${#var} counts codepoints (more precisely,
bytes b with (b & 0xc0) != 0x80).
2011-05-07 14:32:16 +00:00
jilles
6672b66ab7 sh: Track if the current locale's charset is UTF-8 or not. 2011-05-06 22:31:27 +00:00
jilles
e4f71c5640 sh: Change the CTL* bytes to ones invalid in UTF-8.
This ensures that mbrtowc(3) can be used directly once it has been verified
that there is no CTL* byte. Dealing with a CTLESC byte within a multibyte
character would be complicated.

The new values do occur in iso-8859-* encodings. This decreases efficiency
slightly but should not affect correctness.

Caveat: Updating across this change and rebuilding without cleaning may
yield a subtly broken sh binary. By default, make buildworld will clean and
avoid problems.
2011-05-06 20:45:50 +00:00
jilles
5a49f52603 sh: Add $'quoting' (C-style escape sequences).
A string between $' and ' may contain backslash escape sequences similar to
the ones in a C string constant (except that a single-quote must be escaped
and a double-quote need not be). Details are in the sh(1) man page.

This construct is useful to include unprintable characters, tabs and
newlines in strings; while this can be done with a command substitution
containing a printf command, that needs ugly workarounds if the result is to
end with a newline as command substitution removes all trailing newlines.

The construct may also be useful in future to describe unprintable
characters without needing to write those characters themselves in 'set -x',
'export -p' and the like.

The implementation attempts to comply to the proposal for the next issue of
the POSIX specification. Because this construct is not in POSIX.1-2008,
using it in scripts intended to be portable is unwise.

Matching the minimal locale support in the rest of sh, the \u and \U
sequences are currently not useful.

Exp-run done by: pav (with some other sh(1) changes)
2011-05-05 20:55:55 +00:00
jilles
ba0d29571f sh: Apply set -u to variables in arithmetic.
Note that this only applies to variables that are actually used.
Things like (0 && unsetvar) do not cause an error.

Exp-run done by: pav (with some other sh(1) changes)
2011-05-04 22:12:22 +00:00
jilles
fa0f3c42ef sh: Detect an error for ${#var<GARBAGE>}.
In particular, this makes things like ${#foo[0]} and ${#foo[@]} errors
rather than silent equivalents of ${#foo}.

PR:		bin/151720
Submitted by:	Mark Johnston
Exp-run done by: pav (with some other sh(1) changes)
2011-05-04 21:49:34 +00:00
jilles
7b50330e01 sh: Set $? to 0 for background commands.
For backgrounded pipelines and subshells, the previous value of $? was being
preserved, which is incorrect.

For backgrounded simple commands containing a command substitution, the
status of the last command substitution was returned instead of 0.

If fork() fails, this is an error.
2011-04-25 20:54:12 +00:00
jilles
836a99923b sh: Check setuid()/setgid() return values.
If the -p option is turned off, privileges from a setuid or setgid binary
are dropped. Make sure to check if this succeeds. If it fails, this is an
error which will cause the shell to abort except in interactive mode or if
'command' was used to make 'set' or an outer 'eval' or '.' non-special.

Note that taking advantage of this feature and writing setuid shell scripts
seems unwise.

MFC after:	1 week
2011-04-25 10:14:29 +00:00
jilles
54847e6220 sh: Remove duplicate code resetting uid/gid for set +p/+o privileged.
MFC after:	1 week
2011-04-25 10:08:34 +00:00
jilles
f250dc2f44 sh: Allow EV_EXIT through function calls, make {...} <redir more consistent.
If EV_EXIT causes an exit, use the exception mechanism to unwind
redirections and local variables. This way, if the final command is a
redirected command, an EXIT trap now executes without the redirections.

Because of these changes, EV_EXIT can now be inherited by the body of a
function, so do so. This means that a function no longer prevents a fork
before an exec being skipped, such as in
  f() { head -1 /etc/passwd; }; echo $(f)

Wrapping a single builtin in a function may still cause an otherwise
unnecessary fork with command substitution, however.

An exit command or -e failure still invokes the EXIT trap with the
original redirections and local variables in place.

Note: this depends on SHELLPROC being gone. A SHELLPROC depended on
keeping the redirections and local variables and only cleaning up the
state to restore them.
2011-04-23 22:28:56 +00:00
jilles
1347144ea4 sh: Do not word split "${#parameter}".
This is only a problem if IFS contains digits, which is unusual but valid.

Because of an incorrect fix for PR bin/12137, "${#parameter}" was treated
as ${#parameter}. The underlying problem was that "${#parameter}"
erroneously added CTLESC bytes before determining the length. This
was properly fixed for PR bin/56147 but the incorrect fix was not backed
out.

Reported by:	Seeker on forums.freebsd.org
MFC after:	2 weeks
2011-04-20 22:24:54 +00:00
jilles
f4c860e408 sh(1): Describe subshell environment, command substitution more correctly.
POSIX does not require the shell to fork for a subshell environment, and we
use that possibility in various ways (command substitutions with a single
command and most subshells that are the final command of a shell process).
Therefore do not tie subshells to forking in the man page.

Command substitutions with expansions are a bit strange, causing a fork for
$(...$(($x))...) because $x might expand to y=2; they will probably be
changed later but this is how they work now.
2011-03-20 23:52:45 +00:00
jilles
161663c247 sh: Fix some parameter expansion variants ${#...}.
These already worked: $# ${#} ${##} ${#-} ${#?}
These now work as well: ${#+word} ${#-word} ${##word} ${#%word}

There is an ambiguity in the standard with ${#?}: it could be the length of
$? or it could be $# giving an error in the (impossible) case that it is not
set. We continue to use the former interpretation as it seems more useful.
2011-03-13 20:02:39 +00:00
stefanf
0d4be9304a Remove unnecessary cast.
Reviewed by:	jilles
2011-03-07 07:31:15 +00:00
jilles
75dda0ff36 sh(1): Reduce excessive semicolon-separated sentences.
Reported by:	Benjamin Kaduk
2011-03-06 21:20:53 +00:00
jilles
1a2c2ccf00 sh: Fix some warnings in code for arithmetic expressions.
Submitted by:	eadler
2011-03-05 13:27:13 +00:00
brucec
6d9b42b486 Fix typos - remove duplicate "the".
PR:	bin/154928
Submitted by:	Eitan Adler <lists at eitanadler.com>
MFC after: 	3 days
2011-02-21 09:01:34 +00:00
jilles
2fb0603686 sh: Detect dividing the smallest integer by -1.
This overflows and on some architectures such as amd64 it generates SIGFPE.
Generate an error on all architectures.
2011-02-12 23:44:05 +00:00
jilles
a0549c0f22 sh(1): Update description of arithmetic. 2011-02-08 23:19:40 +00:00
jilles
1cbab8a321 sh: Import arithmetic expression code from dash.
New features:
* proper lazy evaluation of || and &&
* ?: ternary operator
* executable is considerably smaller (8K on i386) because lex and yacc are
  no longer used

Differences from dash:
* arith_t instead of intmax_t
* imaxdiv() not used
* unset or null variables default to 0
* let/exp builtin (undocumented, will probably be removed later)

Obtained from:	dash
2011-02-08 23:18:06 +00:00
jilles
ff6aee65ce sh: Fix two things about {(...)} <redir:
* In {(...) <redir1;} <redir2, do not drop redir1.
* Maintain the difference between (...) <redir and {(...)} <redir:
  In (...) <redir, the redirection is performed in the child, while in
  {(...)} <redir it should be performed in the parent (like {(...); :;}
  <redir)
2011-02-05 15:02:19 +00:00
jilles
9a75a8c404 sh: Remove clearcmdentry()'s now unused argument. 2011-02-05 14:08:51 +00:00
jilles
852a80acf7 sh: Forget all cached command locations on any PATH change.
POSIX requires this and it is simpler than the previous code that remembered
command locations when appending directories to PATH.

In particular,
  PATH=$PATH
is no longer a no-op but discards all cached command locations.
2011-02-05 14:01:46 +00:00
jilles
a81357fbe9 sh: Do not try to execute binary files as scripts.
If execve() returns an [ENOEXEC] error, check if the file is binary before
trying to execute it using sh. A file is considered binary if at least one
of the first 256 bytes is '\0'.

In particular, trying to execute ELF binaries for the wrong architecture now
fails with an "Exec format error" message instead of syntax errors and
potentially strange results.
2011-02-05 12:54:59 +00:00
jilles
95ad413d4a sh: Remove special code for shell scripts without magic number.
These are called "shell procedures" in the source.

If execve() failed with [ENOEXEC], the shell would reinitialize itself
and execute the program as a script. This requires a fair amount of code
which is not frequently used (most scripts have a #! magic number).
Therefore just execute a new instance of sh (_PATH_BSHELL) to run the
script.
2011-02-04 22:47:55 +00:00
jilles
dbecc33067 Make sys_signame upper case.
This matches the constants from <signal.h> with 'SIG' removed, which POSIX
requires kill and trap to accept and 'kill -l' to write.

'kill -l', 'trap', 'trap -l' output is now upper case.

In Turkish locales, signal names with an upper case 'I' are now accepted,
while signal names with a lower case 'i' are no longer accepted, and the
output of 'killall -l' now contains proper capital 'I' without dot instead
of a dotted capital 'I'.
2011-02-04 16:40:50 +00:00
jilles
86ccb3f9c0 sh: Return only 126 or 127 for execve() failures.
Do not return 2 for errors other than [EACCES] or [ENOENT].
2011-02-03 23:38:11 +00:00
jilles
e252925aeb sh: Remove comment mentioning herefd, which is gone. 2011-02-02 21:48:53 +00:00
jilles
8605caacbf sh: Send messages about signals to stderr.
This is required by POSIX and seems to make more sense.

See also r217557.
2011-01-30 22:57:52 +00:00
jilles
a123f0aac0 sh: Clean up some old comments:
* There is no plan for an alternative to the command "set".
* Attempting to unset a readonly variable has not raised an error for quite
  a while, so the order of unsetting a variable and a function with the same
  name does not matter.

MFC after:	1 week
2011-01-25 20:56:18 +00:00
jilles
460d7b088e sh: Fix signal messages being sent to the wrong file sometimes.
When a foreground job exits on a signal, a message is printed to stdout
about this. The buffer was not flushed after this which could result in the
message being written to the wrong file if the next command was a builtin
and had stdout redirected.

Example:
  sh -c 'kill -9 $$'; : > foo; echo FOO:; cat foo

Reported by:	gcooper
MFC after:	1 week
2011-01-18 21:18:31 +00:00
jilles
48fcfccda6 sh(1): Document changes to 'exit' from traps. 2011-01-16 14:11:50 +00:00
jilles
3967e15d57 sh: If exit is used without args from a trap action, exit on the signal.
This is useful so that it is easier to exit on a signal than to reset the
trap to default and resend the signal. It matches ksh93. POSIX says that
'exit' without args from a trap action uses the exit status from the last
command before the trap, which is different from 'exit $?' and matches this
if the previous command is assumed to have exited on the signal.

If the signal is SIGSTOP, SIGTSTP, SIGTTIN or SIGTTOU, or if the default
action for the signal is to ignore it, a normal _exit(2) is done with exit
status 128+signal_number.
2011-01-16 13:56:41 +00:00
jilles
31120cf045 sh: Fix some things about -- in trap:
* Make 'trap --' do the same as 'trap' instead of nothing.
* Make '--' stop option processing (note that '-' action is not an option).

Side effect: The error message for an unknown option is different.
2011-01-15 21:09:00 +00:00
jilles
085a83f669 sh: Make 'trap -l' look like 'kill -l'. 2011-01-14 21:30:27 +00:00
jilles
ebbca2a885 sh: Follow-up to r216743, grabstackblock() can be replaced with stalloc().
grabstackblock() was used only once (but it is a very often executed piece
of code).
2011-01-09 22:47:58 +00:00
jilles
2a782244a9 sh: Remove special %builtin PATH entry.
All builtins are now always found before a PATH search.

Most ash derivatives have an undocumented feature where the presence of an
entry "%builtin" in $PATH will cause builtins to be checked at that point of
the PATH search, rather than before looking at any directories as documented
in the man page (very old versions do document this feature).

I am removing this feature from sh, as it complicates the code, may violate
expectations (for example, /usr/bin/alias is very close to a forkbomb with
PATH=/usr/bin:%builtin, only /usr/bin/builtin not being another link saves
it) and appears to be unused (all the %builtin google code search finds is
in some sort of ash source code).

Note that aliases and functions took and take precedence above builtins.
Because aliases work on a lexical level they can only ever be overridden on
a lexical level (quoting or preceding 'builtin' or 'command'). Allowing
override of functions via PATH does not really fit in the model of sh and it
would work differently from %builtin if implemented.

Note: POSIX says special builtins are found before functions. We comply to
this because we do not allow functions with the same name as a special
builtin.

Silence from:	freebsd-hackers@ (message sent 20101225)
Discussed with:	dougb
2011-01-09 21:07:30 +00:00
jilles
3a61afec3c sh: Make exit without parameters from EXIT trap POSIX-compliant.
It should use the original exit status, just like falling off the
end of the trap handler.

Outside an EXIT trap, 'exit' is still equivalent to 'exit $?'.
2011-01-08 23:08:13 +00:00
jilles
3c4cff0f35 sh: Do not call exitshell() from evalcommand() unless evalcommand() forked
itself.

This ensures that certain traps caused by builtins are executed.
2011-01-05 23:17:29 +00:00
jilles
9391068711 sh: Check readonly status for assignments on regular builtins.
An error message is written, the builtin is not executed, nonzero exit
status is returned but the shell does not abort.

This was already checked for special builtins and external commands, with
the same consequences except that the shell aborts for special builtins.

Obtained from:	NetBSD
2011-01-01 13:26:18 +00:00
jilles
e3df947be8 sh: Check if dup2 for redirection from/to a file succeeds.
A failure (e.g. caused by ulimit -n being set very low) is a redirection
error.

Example:
  ulimit -n 9; exec 9<.
2010-12-31 18:20:17 +00:00
jilles
ca3118f4ca sh: Avoid side effects from builtins in optimized command substitution.
Change the criterion for builtins to be safe to execute in the same process
in optimized command substitution from a blacklist of only cd, . and eval to
a whitelist.

This avoids clobbering the main shell environment such as by $(exit 4) and
$(set -x).

The builtins jobid, jobs, times and trap can still show information not
available in a child process; this is deliberately permitted. (Changing
traps is not.)

For some builtins, whether they are safe depends on the arguments passed to
them. Some of these are always considered unsafe to keep things simple; this
only harms efficiency a little in the rare case they are used alone in a
command substitution.
2010-12-30 22:33:55 +00:00
jilles
584bccb74d sh: Properly restore exception handler in fc.
If SIGINT arrived at exactly the right moment (unlikely), an exception
handler in a no longer active stack frame would be called.

Because the old handler was not used in the normal path, clang thought it
was a dead value and if an exception happened it would longjmp() to garbage.
This caused builtins/fc1.0 to fail if histedit.c was compiled with clang.

MFC after:	1 week
2010-12-29 19:39:51 +00:00
jilles
74d9b02bb0 sh: Don't do optimized command substitution if expansions have side effects.
Before considering to execute a command substitution in the same process,
check if any of the expansions may have a side effect; if so, execute it in
a new process just like happens if it is not a single simple command.

Although the check happens at run time, it is a static check that does not
depend on current state. It is triggered by:
- expanding $! (which may cause the job to be remembered)
- ${var=value} default value assignment
- assignment operators in arithmetic
- parameter substitutions in arithmetic except ${#param}, $$, $# and $?
- command substitutions in arithmetic

This means that $((v+1)) does not prevent optimized command substitution,
whereas $(($v+1)) does, because $v might expand to something containing
assignment operators.

Scripts should not depend on these exact details for correctness. It is also
imaginable to have the shell fork if and when a side effect is encountered
or to create a new temporary namespace for variables.

Due to the $! change, the construct $(jobs $!) no longer works. The value of
$! should be stored in a variable outside command substitution first.
2010-12-28 21:27:08 +00:00
jilles
713ef02a1f sh: Make expansion errors in optimized command substitution non-fatal.
Command substitutions consisting of a single simple command are executed in
the main shell process but this should be invisible apart from performance
and very few exceptions such as $(trap).
2010-12-28 13:28:24 +00:00
jilles
f6812a9bf2 sh: Simplify "stack string" code slightly.
Maintain a pointer to the end of the stack string area instead of how much
space is left. This simplifies the macros in memalloc.h. The places where
the new variable must be updated are only where the memory area is created,
destroyed or resized.
2010-12-27 22:18:27 +00:00
jilles
e1ab1f8c3c sh: Fix integer overflow check, it checked an uninitialized variable. 2010-12-26 13:41:53 +00:00
jilles
de73f385a5 sh: Allow arbitrary large numbers in CHECKSTRSPACE.
Reduce "stack string" API somewhat and simplify code.
Add a check for integer overflow of the "stack string" length (probably
incomplete).
2010-12-26 13:25:47 +00:00
jilles
dbd8131dd6 sh(1): Explain why it is a bad idea to use aliases in scripts. 2010-12-21 22:48:56 +00:00
jilles
ae2aabc349 sh: Add kill builtin.
This allows specifying a %job (which is equivalent to the corresponding
process group).

Additionally, it improves reliability of kill from sh in high-load
situations and ensures "kill" finds the correct utility regardless of PATH,
as required by POSIX (unless the undocumented %builtin mechanism is used).

Side effect: fatal errors (any error other than kill(2) failure) now return
exit status 2 instead of 1. (This is consistent with other sh builtins, but
not in NetBSD.)

Code size increases about 1K on i386.

Obtained from:	NetBSD
2010-12-21 22:47:34 +00:00
jilles
eb00352e45 sh: Add a function to print warnings (with command name and newline).
This is like error() but without raising an exception.
It is particularly useful as a replacement for the warnx macro in
bltin/bltin.h.
2010-12-21 20:47:06 +00:00
jilles
ccc4611f77 sh: Make warnings in the printf builtin non-fatal, like in the program.
The #define for warnx now behaves much like the libc function (except that
it uses sh command name and output).

Also, it now uses C99 __VA_ARGS__ so there is no need for three different
macros for 0, 1 or 2 parameters.
2010-12-20 23:06:57 +00:00
jilles
84941f8297 sh: arith: Disallow decimal constants starting with 0 (containing 8 or 9).
Constants in arithmetic starting with 0 should be octal only.

This avoids the following highly puzzling result:
  $ echo $((018-017))
  3
by making it an error instead.
2010-12-18 23:03:51 +00:00
uqs
bd917baec5 Remove dead code.
c is assigned 0 and *loc is pointing to NULL, so c!=0 cannot be true,
and dereferencing loc would be a bad idea anyway.

Coverity Prevent:	CID 5113
Reviewed by:		jilles
2010-12-18 22:16:15 +00:00
jilles
da5b058d1d sh: Fix corruption of command substitutions with special chars after newline
The CTLESC byte to protect a special character was output before instead of
after a newline directly preceding the special character.

The special handling of newlines is because command substitutions discard
all trailing newlines.
2010-12-16 23:28:20 +00:00
uqs
889baffc86 Remove duplicate check, turning dead code into live code.
Coverity CID:	5114
Reviewed by:	jilles
2010-12-13 10:48:49 +00:00
jilles
9624ca1479 sh: Various simplifications to jobs.c:
* Prefer kill(-X) to killpg(X).
* Remove some dead code.
* No additional SIGINT is needed if int_pending() is already true.

No functional change is intended.
2010-12-12 22:59:34 +00:00
jilles
9daf74d4c8 sh: Remove the herefd hack.
The herefd hack wrote out partial here documents while expanding them. It
seems unnecessary complication given that other expansions just allocate
memory. It causes bugs because the stack is also used for intermediate
results such as arithmetic expressions. Such places should disable herefd
for the duration but not all of them do, and I prefer removing the need for
disabling herefd to disabling it everywhere needed.

Here documents larger than 1024 bytes will use a bit more CPU time and
memory.

Additionally this allows a later change to expand here documents in the
current shell environment. (This is faster for small here documents but also
changes behaviour.)

Obtained from:	dash
2010-12-12 00:07:27 +00:00
jilles
9f0c118349 sh: Replace some macros and repeated code in expand.c with functions.
No functional change is intended, but the binary is about 1K smaller on
i386.
2010-12-11 22:13:29 +00:00