* There is no plan for an alternative to the command "set".
* Attempting to unset a readonly variable has not raised an error for quite
a while, so the order of unsetting a variable and a function with the same
name does not matter.
MFC after: 1 week
When a foreground job exits on a signal, a message is printed to stdout
about this. The buffer was not flushed after this which could result in the
message being written to the wrong file if the next command was a builtin
and had stdout redirected.
Example:
sh -c 'kill -9 $$'; : > foo; echo FOO:; cat foo
Reported by: gcooper
MFC after: 1 week
This is useful so that it is easier to exit on a signal than to reset the
trap to default and resend the signal. It matches ksh93. POSIX says that
'exit' without args from a trap action uses the exit status from the last
command before the trap, which is different from 'exit $?' and matches this
if the previous command is assumed to have exited on the signal.
If the signal is SIGSTOP, SIGTSTP, SIGTTIN or SIGTTOU, or if the default
action for the signal is to ignore it, a normal _exit(2) is done with exit
status 128+signal_number.
* Make 'trap --' do the same as 'trap' instead of nothing.
* Make '--' stop option processing (note that '-' action is not an option).
Side effect: The error message for an unknown option is different.
All builtins are now always found before a PATH search.
Most ash derivatives have an undocumented feature where the presence of an
entry "%builtin" in $PATH will cause builtins to be checked at that point of
the PATH search, rather than before looking at any directories as documented
in the man page (very old versions do document this feature).
I am removing this feature from sh, as it complicates the code, may violate
expectations (for example, /usr/bin/alias is very close to a forkbomb with
PATH=/usr/bin:%builtin, only /usr/bin/builtin not being another link saves
it) and appears to be unused (all the %builtin google code search finds is
in some sort of ash source code).
Note that aliases and functions took and take precedence above builtins.
Because aliases work on a lexical level they can only ever be overridden on
a lexical level (quoting or preceding 'builtin' or 'command'). Allowing
override of functions via PATH does not really fit in the model of sh and it
would work differently from %builtin if implemented.
Note: POSIX says special builtins are found before functions. We comply to
this because we do not allow functions with the same name as a special
builtin.
Silence from: freebsd-hackers@ (message sent 20101225)
Discussed with: dougb
It should use the original exit status, just like falling off the
end of the trap handler.
Outside an EXIT trap, 'exit' is still equivalent to 'exit $?'.
An error message is written, the builtin is not executed, nonzero exit
status is returned but the shell does not abort.
This was already checked for special builtins and external commands, with
the same consequences except that the shell aborts for special builtins.
Obtained from: NetBSD
Change the criterion for builtins to be safe to execute in the same process
in optimized command substitution from a blacklist of only cd, . and eval to
a whitelist.
This avoids clobbering the main shell environment such as by $(exit 4) and
$(set -x).
The builtins jobid, jobs, times and trap can still show information not
available in a child process; this is deliberately permitted. (Changing
traps is not.)
For some builtins, whether they are safe depends on the arguments passed to
them. Some of these are always considered unsafe to keep things simple; this
only harms efficiency a little in the rare case they are used alone in a
command substitution.
If SIGINT arrived at exactly the right moment (unlikely), an exception
handler in a no longer active stack frame would be called.
Because the old handler was not used in the normal path, clang thought it
was a dead value and if an exception happened it would longjmp() to garbage.
This caused builtins/fc1.0 to fail if histedit.c was compiled with clang.
MFC after: 1 week
Before considering to execute a command substitution in the same process,
check if any of the expansions may have a side effect; if so, execute it in
a new process just like happens if it is not a single simple command.
Although the check happens at run time, it is a static check that does not
depend on current state. It is triggered by:
- expanding $! (which may cause the job to be remembered)
- ${var=value} default value assignment
- assignment operators in arithmetic
- parameter substitutions in arithmetic except ${#param}, $$, $# and $?
- command substitutions in arithmetic
This means that $((v+1)) does not prevent optimized command substitution,
whereas $(($v+1)) does, because $v might expand to something containing
assignment operators.
Scripts should not depend on these exact details for correctness. It is also
imaginable to have the shell fork if and when a side effect is encountered
or to create a new temporary namespace for variables.
Due to the $! change, the construct $(jobs $!) no longer works. The value of
$! should be stored in a variable outside command substitution first.
Command substitutions consisting of a single simple command are executed in
the main shell process but this should be invisible apart from performance
and very few exceptions such as $(trap).
Maintain a pointer to the end of the stack string area instead of how much
space is left. This simplifies the macros in memalloc.h. The places where
the new variable must be updated are only where the memory area is created,
destroyed or resized.
This allows specifying a %job (which is equivalent to the corresponding
process group).
Additionally, it improves reliability of kill from sh in high-load
situations and ensures "kill" finds the correct utility regardless of PATH,
as required by POSIX (unless the undocumented %builtin mechanism is used).
Side effect: fatal errors (any error other than kill(2) failure) now return
exit status 2 instead of 1. (This is consistent with other sh builtins, but
not in NetBSD.)
Code size increases about 1K on i386.
Obtained from: NetBSD
The #define for warnx now behaves much like the libc function (except that
it uses sh command name and output).
Also, it now uses C99 __VA_ARGS__ so there is no need for three different
macros for 0, 1 or 2 parameters.
Constants in arithmetic starting with 0 should be octal only.
This avoids the following highly puzzling result:
$ echo $((018-017))
3
by making it an error instead.
c is assigned 0 and *loc is pointing to NULL, so c!=0 cannot be true,
and dereferencing loc would be a bad idea anyway.
Coverity Prevent: CID 5113
Reviewed by: jilles
The CTLESC byte to protect a special character was output before instead of
after a newline directly preceding the special character.
The special handling of newlines is because command substitutions discard
all trailing newlines.
* Prefer kill(-X) to killpg(X).
* Remove some dead code.
* No additional SIGINT is needed if int_pending() is already true.
No functional change is intended.
The herefd hack wrote out partial here documents while expanding them. It
seems unnecessary complication given that other expansions just allocate
memory. It causes bugs because the stack is also used for intermediate
results such as arithmetic expressions. Such places should disable herefd
for the duration but not all of them do, and I prefer removing the need for
disabling herefd to disabling it everywhere needed.
Here documents larger than 1024 bytes will use a bit more CPU time and
memory.
Additionally this allows a later change to expand here documents in the
current shell environment. (This is faster for small here documents but also
changes behaviour.)
Obtained from: dash
The code to translate the internal representation to text did not know about
various additions to the internal representation since the original ash and
therefore wrote binary stuff to the terminal.
The code is used in the jobs command and similar output.
Note that the output is far from complete and mostly serves for recognition
purposes.
If describing the status of a pipeline, write all elements of the pipeline
and show the status of the last process (which would also end up in $?).
Only write one report per job, not one for every process that exits.
To keep some earlier behaviour, if any process started by the shell in a
foreground job terminates because of a signal, write a message about the
signal (at most one message per job, however).
Also, do not write messages about signals in the wait builtin in
non-interactive shells. Only true foreground jobs now write such messages
(for example, "Terminated").
In r208489, I added code to reap zombies when forking new processes, to
limit the amount of zombies. However, this can lead to marking a job as done
or stopped if it consists of multiple processes and the first process ends
very quickly. Fix this by only checking for zombies before forking the first
process of a job and not marking any jobs without processes as done or
stopped.
The getpgid() call will fail if the first process in the job has already
terminated, resulting in output of "-1".
The pgid of a job is always the pid of the first process in the job and
other code already relies on this.
Make sure all built-in commands are in the subsection named such, except
exp, let and wordexp which are deliberately undocumented. The text said only
built-ins that really need to be a built-in were documented there but in
fact almost all of them were already documented.
* Prefer one CHECKSTRSPACE with multiple USTPUTC to multiple STPUTC.
* Add STPUTS macro (based on function) and use it instead of loops that add
nul-terminated strings to the stack string.
No functional change is intended, but code size is about 1K less on i386.
If getcwd fails, do not treat this as an error, but print a warning and
unset PWD. This is similar to the behaviour when starting the shell in a
directory whose name cannot be determined.
Since is_alpha/is_name/is_in_name were made ASCII-only, this can no longer
happen.
Additionally, the check was wrong because it did not include the new
CTLQUOTEEND.
This was removed in 2001 but I think it is appropriate to add it back:
* I do not want to encourage people to write fragile and non-portable echo
commands by making printf much slower than echo.
* Recent versions of Autoconf use it a lot.
* Almost no software still wants to support systems that do not have
printf(1) at all.
* In many other shells printf is already a builtin.
Side effect: printf is now always the builtin version (which behaves
identically to /usr/bin/printf) and cannot be overridden via PATH (except
via the undocumented %builtin mechanism).
Code size increases about 5K on i386. Embedded folks might want to replace
/usr/bin/printf with a hard link to /usr/bin/alias.
The information in sh(1) about the echo builtin is equivalent, though less
extensive.
The echo(1) man page (bin/echo/echo.1) is different.
Unfortunately, sh's echo builtin and /bin/echo have gone out of sync and
this probably cannot be fixed any more.
Reported by: uqs (list of untouched files)
MFC after: 1 week
In particular, remove the text about ksh-like features, which are usually
taken for granted nowadays. The original Bourne shell is fading away and for
most users our /bin/sh is one of the most minimalistic they know.
Convert the tests to the perl prove format.
Remove obsolete TEST.README (results of an old TEST.sh for some old Unices)
and TEST.csh (old tests without correct values, far less complete than
TEST.sh).
MFC after: 1 week
This moves the function of the noaliases variable into the checkkwd
variable. This way it is properly reset on errors and aliases can be used
normally in the commands for each case (the case labels recognize the
keyword esac but no aliases).
The new code is clearer as well.
Obtained from: dash
I've noticed various terminal emulators that need to obtain a sane
default termios structure use very complex `hacks'. Even though POSIX
doesn't provide any functionality for this, extend our termios API with
cfmakesane(3), which is similar to the commonly supported cfmakeraw(3),
except that it fills the termios structure with sane defaults.
Change all code in our base system to use this function, instead of
depending on <sys/ttydefaults.h> to provide TTYDEF_*.
These do something else in ksh: name=(...) is an array or compound variable
assignment and the others are extended patterns.
This is the last patch of the ones tested in the exp run.
Exp-run done by: pav (with some other sh(1) changes)
Apart from detecting breakage earlier or at all, this also fixes a segfault
in the testsuite. The "handling" of the breakage left an invalid internal
representation in some cases.
Examples:
echo a; do echo b
echo `) echo a`
echo `date; do do do`
Exp-run done by: pav (with some other sh(1) changes)
subevalvar() incorrectly assumed that CTLESC bytes were present iff the
expansion was quoted. However, they are present iff various processing such
as word splitting is to be done later on.
Example:
v=@$e@$e@$e@
y="${v##*"$e"}"
echo "$y"
failed if $e contained the magic CTLESC byte.
Exp-run done by: pav (with some other sh(1) changes)
The code is inspired by NetBSD sh somewhat, but different because we
preserve the old Almquist/Bourne/Korn ability to have an unquoted part in a
quoted ${v+word}. For example, "${v-"*"}" expands to $v as a single field if
v is set, but generates filenames otherwise.
Note that this is the only place where we split text literally from the
script (the similar ${v=word} assigns to v and then expands $v). The parser
must now add additional markers to allow the expansion code to know whether
arbitrary characters in substitutions are quoted.
Example:
for i in ${$+a b c}; do echo $i; done
Exp-run done by: pav (with some other sh(1) changes)
If double-quote state does not match, treat the '}' literally.
This ensures double-quote state remains the same before and after a
${v+-=?...} which helps with expand.c.
It makes things like
${foo+"\${bar}"}
which I have seen in the wild work as expected.
Exp-run done by: pav (with some other sh(1) changes)
This is a syntax error.
POSIX does not say explicitly whether defining a function with the same name
as a special builtin is allowed, but it does say that it is impossible to
call such a function.
A special builtin can still be overridden with an alias.
This commit is part of a set of changes that will ensure that when
something looks like a special builtin to the parser, it is one. (Not the
other way around, as it remains possible to call a special builtin named
by a variable or other substitution.)
Exp-run done by: pav (with some other sh(1) changes)
Add some conservative checks on function names:
- Disallow expansions or quoting characters; these can only be called via
strange control characters
- Disallow '/'; these functions cannot be called anyway, as exec.c assumes
they are pathnames
- Make the CTL* bytes work properly in function names.
These are syntax errors.
POSIX does not require us to support more than names (letters, digits and
underscores, not starting with a digit), but I do not want to restrict it
that much at this time.
Exp-run done by: pav (with some other sh(1) changes)
This is how ksh93 treats ! within a pipeline and makes the ! in
a | ! b | c
negate the exit status of the pipeline, as if it were
a | { ! b | c; }
Side effect: something like
f() ! a
is now a syntax error, because a function definition takes a command,
not a pipeline.
Exp-run done by: pav (with some other sh(1) changes)
For multi-command pipelines,
1. all commands are direct children of the shell (unlike the original
Bourne shell)
2. all commands are executed in a subshell (unlike the real Korn shell)
MFC after: 1 week
immediately written into the stack after the call. Instead let the caller
manage the "space left".
Previously, growstackstr()'s assumption causes problems with STACKSTRNUL()
where we want to be able to turn a stack into a C string, and later
pretend the NUL is not there.
This fixes a bug in STACKSTRNUL() (that grew the stack) where:
1. STADJUST() called after a STACKSTRNUL() results in an improper adjust.
This can be seen in ${var%pattern} and ${var%%pattern} evaluation.
2. Memory leak in STPUTC() called after a STACKSTRNUL().
Reviewed by: jilles
- Use %t to print ptrdiff_t values.
- Cast a ptrdiff_t value explicitly to int for a field width specifier.
While here, sort includes.
Submitted by: Garrett Cooper
frobbing CFLAGS directly. DEBUG_FLAGS is something that can be specified
on the make command line without having to edit the Makefile directly.
Submitted by: Garrett Cooper
Add directory names directly and sort at the end.
Include bsd.arch.inc.mk so we can, in the future, more easily make arch
dependent changes in /bin (unlikely, but is needed for symmetry).
expr(1) should usually not be used as various forms of parameter expansion
and arithmetic expansion replicate most of its functionality in an easier
way.
getopt(1) should not be used at all in new code. Instead, getopts(1) or
entirely manual parsing should be used.
MFC after: 1 week
The three examples are better done using sh(1) itself these days.
The example
expr -- "$a" : ".*"
is incorrect in the general case, as "$a" may be an operator.
MFC after: 2 weeks
This makes it impossible to use locale-specific characters in variable
names.
Names containing locale-specific characters make scripts only work with the
correct locale setting. Also, they did not even work in many practical cases
because multibyte character sets such as utf-8 are not supported.
This also avoids weirdness if LC_CTYPE is changed in the middle of a script.
are too long. Filenames escaping this test are caught later on,
so the bug doesn't cause any breakage.
Document the correct ustar limitations in pax. As I have no access
to the IEEE 1003.2 spec, I can only assume that the limitations
imposed are in fact correct.
Add regression tests for the filename limitations imposed by pax.
MFC after: 3 weeks
This Almquist extension was disabled long ago.
In pathname generation, components starting with '!!' were treated as
containing wildcards, causing unnecessary readdir (which could fail, causing
pathname generation to fail while it should not).
In our implementation and most others, a break or continue in a dot script
can break or continue a loop outside the dot script. This should cause all
further commands in the dot script to be skipped. However, cmdloop() did not
know about this and continued to parse and execute commands from the dot
script.
As described in the man page, a return in a dot script in a function returns
from the function, not only from the dot script. There was a similar issue
as with break and continue. In various other shells, the return appears to
return from the dot script, but POSIX seems not very clear about this.
The buffer for generated pathnames could be too small in some cases. It
happened to be always at least PATH_MAX long, so there was never an overflow
if the resulting pathnames would be usable.
This bug may be abused if a script subjects input from an untrusted source
to pathname generation, which a bad idea anyhow. Most shell scripts do not
work on untrusted data. secteam@ says no advisory is necessary.
PR: bin/148733
Reported by: Changming Sun snnn119 at gmail com
MFC after: 10 days
This makes a difference if there is a command substitution.
To make this work, evalstring() has been changed to set exitstatus to 0 if
no command was executed (the string contained only whitespace).
Example:
eval $(false); echo $?
should print 0.
refusing to use stdio.
Reduce nesting level in the sleep loop by returning earlier for negative
timeouts.
Limit the maximum timeout to INT_MAX seconds.
Submitted by: bde
MFC after: 3 weeks
This simply sets a flag in libedit. It has a shortcoming in that it does not
apply to multi-line commands.
Note that a configuration option for this is not going to happen, but always
having this seems better than not having it. NetBSD has done the same.
PR: bin/54683
Obtained from: NetBSD
MFC after: 1 month
So a command like
kill _HUP 1
now fails without sending SIGTERM to init.
The behaviour when kill(2) fails remains unchanged: processing continues.
This matches other implementations and POSIX and is useful for killing
multiple processes at once when some of them may already be gone.
PR: bin/40282
If an ; or & token was followed by an EOF token, pending here-documents were
left uninitialized. Execution would crash, either in the main shell process
for literal here-documents or in a child process for expanded
here-documents. In the latter case the problem is hard to detect apart from
the core dumps and log messages.
Side effect: slightly different retries on inputs where EOF is not
persistent.
Note that tools/regression/bin/sh/parser/heredoc6.0 still causes a similar
crash in a child process. The text passed to eval is malformed and should be
rejected.
simplecmd() only handles simple commands and function definitions, neither
of which involves the ! keyword. The initial token on entry to simplecmd()
is one of the following: TSEMI, TAND, TOR, TNL, TEOF, TWORD, TRP.
Unless $! has been referenced for a particular job or $! still contains that
job's pid, forget about it after it has terminated. If $! has been
referenced, remember the job until the wait builtin has reported its
completion (either with the pid as parameter or without parameters).
In interactive mode, jobs are forgotten after termination has been reported,
which happens before primary prompts and through the jobs builtin. Even
then, though, remember a job if $! has been referenced.
This is similar to what is suggested by POSIX and should fix most memory
leaks (which also tend to cause sh to use more CPU time) with long running
scripts that start background jobs.
Caveats:
* Repeatedly referencing $! without ever doing 'wait', like
while :; do foo & echo started foo: $!; sleep 60; done
will still use a lot of memory and CPU time in the long run.
* The jobs and jobid builtins do not cause a job to be remembered for longer
like expanding $! does.
PR: bin/55346
The LINENO code uses snprintf() and relied on "myhistedit.h" to pull in the
necessary <stdio.h>.
Compiling with -DNO_HISTORY disables all editing and history support and
allows linking without -ledit -ltermcap. This may be useful for embedded
systems.
MFC after: 2 weeks
This uses the new libedit completion function with quoting support.
Unlike NetBSD, there is no 'set +o tabcomplete' option to disable
completion. I do not see any reason for such a special treatment, as
completion is rather useful and it is possible to do
bind ^I ed-insert
to disable completion and insert a tab character instead.
Submitted by: Guy Yur
- .Nd in section NAME is not optional
- .Ed was missing
- "indent" is not a flag, but a literal argument for -offset
- stop switching font sizes for acronyms
- use .Brq instead of rolling our own
is enabled.
This already worked if without job control.
In either case, this depends on it that a process that terminates due to
SIGINT exits on it (so not with status 1, or worse, 0).
Example:
sleep 5; echo continued
This does not print "continued" any more if sleep is aborted via ctrl+c.
MFC after: 1 month
Previously, it would either try to copy it anyway and fail (without -R),
or create fifo instead of the socket (with -R).
Found with: Coverity Prevent
CID: 5623
MFC after: 2 weeks
usually be set first when using -v.
Adjust an example that sets the day to 30 before setting the month to 3 in
accordance with this approach as the example would always fail in February!
PR: 147354
MFC after: 2 weeks
Example (in interactive mode):
cat <<EOF && )
The next command typed caused sh to segfault, because the state for the here
document was not reset.
Like parser_temp, this uses the fact that the parser is not re-entered.
If a command substitution contains a newline token, this no longer starts
here documents of outer commands. This way, we follow POSIX's idea of the
command substitution being a separate script more closely. It also matches
other shells better and is consistent with newline characters in quotes not
starting here documents.
The extension tested in parser/heredoc3.0 ($(cat <<EOF)\ntext\nEOF\n)
continues to be supported.
In particular, this change allows things like
cat <<EOF && echo `pwd`
(a `` command substitution after a here document)
which formerly silently used an empty file as the here document, because the
EOF of the inner command "pwd" also forced an empty here document.
Although "--" historically has not been required to be recognized for
certain special builtins that do not take options in POSIX, some other
implementations recognize options for them, requiring scripts to use "--" or
avoid operands starting with "-".
Operands starting with "-" can be avoided with eval by prepending a space,
and cannot occur with break, continue, exit, return and shift as they only
take numbers, nor with times as it does not take operands. With . and exec,
avoiding "-" is not so easy as it may require reimplementing the PATH
search; therefore the current proposal for POSIX is to require recognition
of "--" for them.
We continue to accept other strings starting with "-" as operands to . and
exec, and also "--" if it is alone to . (which would otherwise be invalid
anyway).
* Move the "environment variables" that do not need exporting to be
effective or that are set by the shell without exporting to a new section
"Special Variables".
* Add special variables LINENO and PPID.
* Add environment variables LANG, LC_* and PWD; also describe ENV under
environment variables.
This prevents accumulating huge amounts of zombies if a script executes
many background commands but no external commands or subshells.
Note that zombies will not be reaped during long calculations (within
the shell process) or read builtins, but those actions do not create
more zombies.
The terminated background commands will also still be remembered by the
shell.
PR: bin/55346
pax(1) was trying to copy the back-referenced data from
the match pattern, not the matched data.
PR: bin/118132
Obtained from: Debian bug #451361
Reviewed by: jilles
MFC after: 3 weeks
These are git commits 36f0fa8fcbc8c7b2b194addd29100fb40e73e4e9 and
d6d06ff5c2ea0fa44becc5ef4340e5f2f15073e4 in dash.
Because this is the first code I'm importing from dash to expand.c, add the
Herbert Xu copyright notice which is in dash's expand.c.
When pathname expanding *\/, the CTLESC representing the quoted state was
erroneously taken as part of the * pathname component. This CTLESC was then
seen by the pattern matching code as escaping the '\0' terminating the
string.
The code is slightly different because dash converts the CTLESC characters
to backslashes and removes all the other CTL* characters to allow
substituting glob(3).
The effect of the bug was also slightly different from dash (where nothing
matched at all). Because a CTLESC can escape a '\0' in some way, whether
files were included despite the bug depended on memory that should not be
read. In particular, on many machines /*\/ expanded to a strict subset of
what /*/ expanded to.
Example:
echo /*"/null"
This should print /dev/null, not /*/null.
PR: bin/146378
Obtained from: dash
This allows doing things like LC_ALL=C some_builtin to run a builtin under a
different locale, just like is possible with external programs. The
immediate reason is that this allows making printf(1) a builtin without
breaking things like LC_NUMERIC=C printf '%f\n' 1.2
This change also affects special builtins, as even though the assignment is
persistent, the export is only to the builtin (unless the variable was
already exported).
Note: for this to work for builtins that also exist as external programs
such as /bin/test, the setlocale() call must be under #ifndef SHELL. The
shell will do the setlocale() calls which may not agree with the environment
variables.
in at least three ways, so do not say it is ignored:
* who may delete/rename a symlink in a sticky directory
* who may do lchflags(2)/lchown(2)/lchmod(2)
* whose inode quota is charged
MFC after: 1 week
In the 'ln source... directory' synopsis, the basename of each source
determines the name of the created link. Determine this using basename(3)
instead of strrchr(..., '/') which is incorrect if the pathname ends in a
slash.
The patch is somewhat changed to allow for basename(3) implementations that
change the passed pathname, and to fix the -w option's checking also.
The code to compare directory entries only applies to hard links, which
cannot be created to directories using ln.
Example:
ln -s /etc/defaults/ /tmp
This should create a symlink named defaults.
PR: 121568
Submitted by: Ighighi
MFC after: 1 week
Two pathnames refer to the same directory entry iff the directories match
and the final components' names match.
Example: (assuming file1 is an existing file)
ln -f file1 file1
This now fails while leaving file1 intact. It used to delete file1 and then
complain it cannot be linked because it is gone.
With -i, this error is detected before the question is asked.
MFC after: 2 weeks
Unset PWD if it is incorrect and no value for it can be determined.
This preserves the logical current directory across shell invocations.
Example (assuming /home is a symlink):
$ cd
$ pwd
/home/foo
$ sh
$ pwd
/home/foo
Formerly the second pwd would show the physical path (symlinks resolved).
Although groff_mdoc(7) gives another impression, this is the ordering
most widely used and also required by mdocml/mandoc.
Reviewed by: ru
Approved by: philip, ed (mentors)
These do pretty much nothing (except that parentheses are ignored), but
people seem to use them and allowing them does not hurt much.
Single-quotes seem not to be used and cause silently different behaviour
with ksh93 character constants.
This makes sh a bit more friendly in single user mode, make buildenv, chroot
and the like, and matches other shells.
The -o emacs can be overridden on the command line or in the ENV file.
Note that the following sentence
> Enclosing the full parameter expansion string in double-quotes does not
> cause the following four varieties of pattern characters to be quoted,
> whereas quoting characters within the braces has this effect.
is now true, but used to be incorrect.
This applies to word in ${v-word}, ${v+word}, ${v=word}, ${v?word} (which
inherits quoting from the outside) and in ${v%word}, ${v%%word}, ${v#word},
${v##word} (which does not inherit any quoting).
In all cases tilde expansion is only attempted at the start of word, even if
word contains spaces. This agrees with POSIX and other shells.
This is the last part of the patch tested in the exp-run.
Exp-run done by: erwin (with some other sh(1) changes)
Note that this depends on r206145 for allowing pattern match characters to
have their special meaning inside a double-quoted expansion like "${v%pat}".
PR: bin/117748
Exp-run done by: erwin (with some other sh(1) changes)
They will be treated like normal characters, resulting in a runtime
arithmetic expression error.
Exp-run done by: erwin (with some other sh(1) changes)
* remove the backslash from \} inside double quotes inside +-=?
substitutions, e.g. "${$+\}a}"
* maintain separate double-quote state for ${v#...} and ${v%...};
single and double quotes are special inside, even in a double-quoted
string or here document
* keep track of correct order of substitutions and arithmetic
This is different from dash's approach, which does not track individual
double quotes in the parser, trying to fix this up during expansion.
This treats single quotes inside "${v#...}" incorrectly, however.
This is similar to NetBSD's approach (as submitted in PR bin/57554), but
recognizes the difference between +-=? and #% substitutions hinted at in
POSIX and is more refined for arithmetic expansion and here documents.
PR: bin/57554
Exp-run done by: erwin (with some other sh(1) changes)
The old approach was wrong because PS2 was not used and seems unlikely to
parse extensions (ksh93's ${ COMMAND} may well fail to parse).
Exp-run done by: erwin (with some other sh(1) changes)
Redirection errors on subshells already did not abort the shell because
the redirection is executed in the subshell.
Other shells seem to agree that these redirection errors should not abort
the shell.
Also ensure that the redirections will be cleaned up properly in cases like
command eval '{ shift x; } 2>/dev/null'
Example:
{ echo bad; } </var/empty/x; echo good
Although simple commands without a command word (only assignments and/or
redirections) are much like special builtins, POSIX and most shells seem to
agree that redirection errors should not abort the shell in this case. Of
course, the assignments persist and assignment errors are fatal.
To get the old behaviour portably, use the ':' special builtin.
To get the new behaviour portably, given that there are no assignments, use
the 'true' regular builtin.
Make parsebackq a function instead of an emulated nested function.
This puts the setjmp usage in a smaller function where it is easier to avoid
bad optimizations.
* avoid unnecessary fork
* allow executing builtins via command
* executing a special builtin via command removes its special properties
Obtained from: NetBSD (parts)
Although argc and argv are never read after the longjmp is complete,
gcc is not clever enough to see that and needlessly warns about it.
So add volatile to silence the compiler.
Approved by: ed (the co-mentor, not ed(1))
Otherwise the -i option will show the inode number of the referenced file
for symbolic links given on the command line. Similarly, the file color
was printed according to the link target in colorized output.
PR: bin/102394
Reviewed by: jilles
MFC after: 2 weeks
All the elements of these structs are char anyway, so it won't hurt
performance.
Bump warns back up to the default.
# we likely should have CTASSERTS to make sure they are the right size.
# but with libarchive based tar maybe we shouldn't bother.
- Allow -h option to work if the listing contains at least one device
file.
- Align major and minor device numbers correctly to the size field.
PR: bin/125678
Approved by: trasz (mentor)
MFC after: 1 month
provides an empty fts_name and reporting the full path is more
appropriate especially with the -R option.
PR: bin/107515
Submitted by: bde
Approved by: trasz (mentor)
MFC after: 1 week
feature parity with du(1) and similar: When set, cp(1) will not traverse
mount points.
Initial patch by: Graham J Lee leeg teaching.physics.ox.ac.uk
PR: bin/88056
Initial patch by: Graham J Lee leeg teaching.physics.ox.ac.uk
Approved by: ed (mentor)
MFC after: 1 month
csh allows you to monitor the utmp(x) file to monitor certain user
logins. Unfortunately it needs to directly stat() this file. I don't
want to break this module , but eventually it shouldn't do that. The
idea of the getutxent(3) API is to hide file access.
Approved by: mp
functionality. Per the regression tests (pgrep-t.t & pkill-t.t), "-t"
should accept "v1", which means a plain number should be accepted for
UNIX98-style PTY's.
- correctly handle error output in $(builtin 2>&1), clarify out1/out2 vs
output/errout in the code
- treat all builtins as regular builtins so errors do not abort the shell
and variable assignments do not persist
- respect the caller's INTOFF
Some bugs still exist:
- expansion errors may still abort the shell
- some side effects of expansions and builtins persist
This will be important when things like 'command eval f' will be possible.
Currently, the funcnest = 0 assignment in RESET (called when returning to
the top level after an error in interactive mode) is really sufficient.
Fix some wrong usages.
Note: this does not affect generated binaries as this argument is not used.
PR: 137213
Submitted by: Eygene Ryabinkin (initial version)
MFC after: 1 month
Reset the exception handler in the child to main's.
This avoids inappropriate double cleanups or shell duplication when the
exception is caught, such as 'fc' and future 'command eval' and 'command .'.
That is, do not do tilde expansion if any of the CTL* bytes (\201-\210), not
only CTLESC and CTLQUOTEMARK, are encountered. Such an expansion would look
up a user name with sh's internal representation.
The parser does not currently distinguish between backslashed and
unbackslashed \201-\210, so tilde expansion of user names with these bytes
in them is not so easy to fix.
It is already done by evalcommand(), unless special-ness has been removed,
in which case variable assignments should not persist. (These are currently
always special builtins, but this will change later: command builtin,
command substitution.)
This also fixes a memory leak when calling . with variable assignments.
Example:
valgrind --leak-check=full sh -c 'x=1 . /dev/null; x=2'
- Redirecting fds that were not open before kept two copies of the
redirected file.
sh -c '{ :; } 7>/dev/null; fstat -p $$; true'
(both fd 7 and 10 remained open)
- File descriptors used to restore things after redirection were not
set close-on-exec, instead they were explicitly closed before executing
a program normally and before executing a shell procedure. The latter
must remain but the former is replaced by close-on-exec.
sh -c 'exec 7</; { exec fstat -p $$; } 7>/dev/null; true'
(fd 10 remained open)
The examples above are simpler than the testsuite because I do not want to
use fstat or procstat in the testsuite.
* exception handlers are now run with interrupts disabled, which avoids
many race conditions
* fix some cases where SIGINT only aborts one command and continues the
script, in particular if a SIGINT causes an EINTR error which trumped the
interrupt.
Example:
sh -c 'echo < /some/fifo; echo This should not be printed'
The fifo should not have writers. When pressing ctrl+c to abort the open,
the shell used to continue with the next command.
Example:
sh -c '/bin/echo < /some/fifo; echo This should not be printed'
Similar. Note, however, that this particular case did not and does not work
in interactive mode with job control enabled.
This avoids weirdness when 'fc -e vi' or the like is done and there is a
syntax error in the file. Formerly an interactive shell tried to execute
stuff after the syntax error and exited.
This should also avoid similar issues with 'command eval' and 'command .'
when 'command' is implemented properly as in NetBSD sh.
Special builtins did not have this problem since errors in them cause the
shell to exit or to reset various state such as the current command input
file.
* increase buffer size from 100 to 256 bytes
* remove implied flush from out2str(), in particular this avoids unnecessary
flushing in the middle of a -x tracing line
* rename dprintf() to out2fmt_flush(), make it flush out2 and use this
function in various places where flushing is desired after an error
message
This is similar to the Solaris utility of the same name.
Some use cases:
* rc.subr's wait_for_pids
* interactive use, e.g. to shut down the computer when some task is done
even if the task is already running
Discussed on: hackers@
This seems more useful and will likely be in the next POSIX standard.
Also document more precisely in the man page what set -u does (note that
$@, $* and $! are the only special parameters that can ever be unset, all
the others are always set, although they may be empty).
the line number where the command substitution started.
This applies to both the $() and `` forms but is most useful for ``
because the other line number is relative to the enclosed text there.
(For older versions, -v can be used as a workaround.)
According to the man page, when neither -H/-L nor -F/-d/-l are given, -H is
implied. This agrees with POSIX, GNU ls and Solaris ls. This means that -p,
although it is very similar to -F, does not prevent the implicit following
of symlinks.
PR: standards/128546
This also fixes that trying to execute a non-regular file with a command
name without '/' returns 127 instead of 126.
The fix is rather simplistic: treat CMDUNKNOWN as if the command were found
as an external program. The resulting fork is a bit wasteful but executing
unknown commands should not be very frequent.
PR: bin/137659
Due to the amount of code removed by this, it seems that allowing unmatched
quotes was a deliberate imitation of System V sh and real ksh. Most other
shells do not allow unmatched quotes (e.g. bash, zsh, pdksh, NetBSD /bin/sh,
dash).
PR: bin/137657
I do not consider this a bug because POSIX permits it and argument strings
and environment variables cannot contain '\0' anyway.
PR: bin/25542
MFC after: 2 weeks
Note that this changes error reporting behaviour somewhat - before,
no error was reported if ACL couldn't be copied because the target
filesystem doesn't support ACLs. Now, it will be reported - of course,
only if there actually is an ACL to copy.
Reviewed by: rwatson
- Keep variables sorted
- Fix logic error with -f and -v options - don't print
the usual -v output if there was an error, whether or not
we were passed -f
- Don't call free(3) just before exit(2)
- Whitespace fixes
Submitted by: bde
Empty pairs of braces are represented by a NULL node pointer, just like
empty lines at the top level.
Support for empty pairs of braces may be removed later. They make the code
more complex, have inconsistent behaviour (may or may not change $?), are
not specified by POSIX and are not allowed by some other shells like bash,
dash and ksh93.
Reported by: kan
Add a reference count to function definitions.
Memory may leak if multiple SIGINTs arrive in interactive mode,
this will be fixed later by changing SIGINT handling.
PR: bin/137640
This implements the POSIX.1-2008 -L and -P flags.
The default remains to create hard links to the target of symlinks.
Approved by: re (kib), ed (mentor)
is identical to the mode computed from that ACL will modify the ACL.
For example, mode computed from the following ACL is 0600:
user:kamila:rwx--------C--:------:allow
owner@:--x-----------:------:deny
owner@:rw-p---A-W-Co-:------:allow
group@:rwxp----------:------:deny
group@:--------------:------:allow
everyone@:rwxp---A-W-Co-:------:deny
everyone@:------a-R-c--s:------:allow
However, applying that mode (chmod 0600) changes the ACL into this:
user:kamila:rwx-----------:------:deny
user:kamila:rwx--------C--:------:allow
owner@:--x-----------:------:deny
owner@:rw-p---A-W-Co-:------:allow
group@:rwxp----------:------:deny
group@:--------------:------:allow
everyone@:rwxp---A-W-Co-:------:deny
everyone@:------a-R-c--s:------:allow
In chmod(1) utility, there is an optimisation, which makes it not
call chmod(2) if the mode of the file is the same as the new mode.
Disable that optimisation for files which may have NFSv4 ACLs.
Reviewed by: rwatson
Approved by: re (kib)
in particular "$@"$ifschar if the final positional parameter is empty.
With the NetBSD code, adding the $ifschar removes a parameter.
PR: standards/79067
Approved by: ed (mentor) (implicit)
(EV_EXIT). The fork is still done as normal if any traps are active.
In many cases, the fork can be avoided even without this change by using {}
instead of (), but in practice many scripts use (), likely because the
syntax is simpler.
Example:
sh -c '(/bin/sleep 10)& sleep 1;ps -p $! -o comm='
Now prints "sleep" instead of "sh". $! is more useful this way.
Most shells (dash, bash, pdksh, ksh93, zsh) seem to print "sleep" for this.
Example:
sh -c '( ( ( (ps jT))))'
Now shows no waiting shell processes instead of four.
Most shells (dash, bash, pdksh, ksh93, zsh) seem to show zero or one.
PR: bin/74404
Approved by: ed (mentor) (implicit)
- update for getrlimit(2) manpage;
- support for setting RLIMIT_SWAP in login class;
- addition to the limits(1) and sh and csh limit-setting builtins;
- tuning(7) documentation on the sysctls controlling overcommit.
In collaboration with: pho
Reviewed by: alc
Approved by: re (kensmith)
- remove ineffective and unnecessary (void) &var; [1]
- remove some unnecessary volatile keywords
- add a necessary volatile keyword
- save the old handler before doing something that could use the saved
value
Submitted by: Christoph Mallon [1]
Approved by: ed (mentor)
Formerly, it was possible for the file to be created between the check if it
existed and the open; the contents would then be lost.
Because this must use O_EXCL, noclobber > will not create a file through a
symlink anymore. This agrees with behaviour of other shells.
Approved by: ed (mentor) (implicit)
Output quoted suitable for re-input to the shell occurs in
various cases such as 'set', 'trap'.
Bugfix: *, ? and [ must be quoted (except sole [)
Bugfix: ~ and # must be quoted (really only sometimes, but keep it simple)
Bugfix: space, tab and newline must always be quoted
Shortening: other IFS characters do not need quoting
Bugfix: send to correct output file, not hard-coded stdout
Shortening: avoid unnecessary '' with \'
Approved by: ed (mentor)
For parsing an old-style backquote substitution (`...`),
a string "file" is used to store the contents of the
substitution (with the special backslash processing done).
If an error occurs, the shell cleans up all these files
(returning to the top level) and flush the top level
file. Erroneously, it first flushed the current file and
then cleaned up all extra files, so that the top level
file (i.e. the terminal) was not flushed.
Example (in interactive mode):
echo `for` echo This should not be printed
Also noticeable in (in interactive mode):
echo `(`
The old version prints an extraneous prompt.
Approved by: ed (mentor)
This change only affects strings passed to -c, when the -s
option is not used.
The approach is to check if there may be additional data
in the string after parsing each command. If there is none,
use the EV_EXIT flag so that a fork may be omitted in
specific cases.
If there are empty lines after the command, the check will
not see the end and forks will not be omitted. The same
thing seems to happen in bash.
Example:
sh -c 'ps lT'
No longer shows a shell process waiting for ps to finish.
PR: bin/113860
Reviewed by: stefanf
Approved by: ed (mentor)
Example:
sh -c '(trap "echo trapped" EXIT; sleep 3)'
now correctly prints "trapped".
With this check, it is no longer necessary to check for -T
explicitly in that case.
This is a useful bugfix by itself and also important because I plan to
skip forking more often.
PR: bin/113860 (part of)
PR: bin/74404 (part of)
Reviewed by: stefanf
Approved by: ed (mentor)
The exit status may exceed 255 in some cases (return); even though it seems
unwise to rely on this, it is also unwise to assume that $? is always
between 0 and 255.
This resolves bin/124748 by documenting that 'exit -1' is not valid.
PR: bin/124748
Approved by: ed (mentor)
Base 10 is always used for the inode counts as I could not think of any
reason base 2 inode counts would be useful.
Minor mdoc markup fix to df(1) while here anyway.
MFC after: 3 weeks
character.
This avoids using non-standard behaviour of the old (upto FreeBSD 7) TTY
layer: it reprocesses the input queue when switching to canonical mode. The
new TTY layer does not provide this functionality and so read -t worked
very poorly (first character is not echoed, cannot be backspaced but is
still read).
This also agrees with what most other shells with read -t do.
PR: bin/129566
Reviewed by: stefanf
Approved by: ed (mentor)
would always terminate if eval returned with a non-zero exit status regardless
if the status was actually tested. Unfortunately a new file-scope variable
is needed, the alternative would only be to add a new parameter to all
built-ins.
PR: 134881
Formerly, this tried to clear the flags on the symlink's target
instead of the symlink itself.
As before, this only happens for root or for the unlink(1) variant of rm.
PR: bin/111226 (part of)
Submitted by: Martin Kammerhofer
Approved by: ed (mentor)
MFC after: 3 weeks
More precisely, this gives precedence to an interpretation not using the
'(', ')', '-a' and '-o' in their special meaning, if possible. For example,
it is now safe to write [ "$a" = "$b" ] and assume it compares the two
strings.
The man page already says that test(1) works this way, so does not need to
be changed.
Interpretation of input with more parameters tries a bit harder to find a
valid parse in some cases.
Add various additional test cases to TEST.sh.
PR: standards/133369
Approved by: ed (mentor)
always surprising when you kill a 'sh -c ...' ancestor or when you kill
yourself when using -f.
Add a -a switch for backwards compatibility.
MFC after: 3 weeks
A difference between the old and the new TTY layer is that the new
implementation does not perform any post-processing before returning
data back to userspace when calling read().
sh(1)'s read turns the TTY into a raw mode before calling select(). This
means that the first character will not receive any ICRNL processing.
Inherit this flag from the original terminal attributes.
Even though this issue is not present on RELENG_*, I'm MFCing it to make
sh(1) in jails behave better.
PR: bin/129566
MFC after: 2 weeks
label is choosen as last printout (ucomm suffers of this such bug
too). That bug is caused by the fact that the fixed size of
printout doesn't leave enough space for them to be printed out.
Implement ucomm and comm commands with a dynamic size lenght for
buffers.
[2] On AMD64 architecture pointers don't have enough chars space to
be shown (8 chars while they need 16). Fix them by providing
a variadic space so that it fits well on both 64 and 32 bits
architectures.
[3] Check a return value of malloc() that wasn't checked before.
PR: bin/128841, bin/128842
Reviewed by: jhb, emaste
Sponsored by: Sandvine Incorporated
symlinks after setting the owner. As a result, mode
and timestamp were not restored. This patch corrects the
problem by simply removing the short-circuit for symlinks
and using lchown()/lchmod()/lutimes() always for restoring
metadata.
PR: bin/91316
Submitted by: Jaakko Heinonen
Reviewed by: Joerg Sonnenberger
MFC after: 14 days
This should be a bit useful for users who look at the manpage and
then try to copy data CD-ROM disks using dd. A lot of us know
that bs=2048 is required, but it still manages to cause a bit of
grief to those who haven't heard about it.
PR: bin/130857
Submitted by: Tri Brotoharsono < mail at tribrotoharsono.net >
MFC after: 3 days
rather than a voltag name, do not set the CESR_VOLTAGS flags in the
CHIOGSTATUS command requesting the current status. As voltags are an
optional feature that must be handled as "reserved" by media changers
not implementing the feature, always setting CESR_VOLTAGS resulted in
the command being aborted with an `Invalid field in CDB', and
consequently the "chio return" failed, for media changers that do not
support voltags.
MFC after: 1 week
in the system. A simple heuristics is used to detect what is "enough"
memory: if number of physmem pages is greater than 32k (equalling 128 MB
on machines with 4 kB pages).
Typical immediate result of these changes is reduction in context switches
and the goal is to increase efficiency by using large buffers:
before: /usr/bin/time -hlp cat file1 > file2
...
163 voluntary context switches
11194 involuntary context switches
after: /usr/bin/time -hlp ./cat file1 > file2
...
417 voluntary context switches
272 involuntary context switches
Reviewed by: hackers@ (no objections to earlier version of cat patch)
Approved by: gnn (mentor)
MFC after: 4 months
In my previous commit I disabled pkill(1)'s automatic prepending of the
"tty" string when `pkill -t' was being used. Re-enable it and stat()
both possible device names when called.
Requested by: jhb, rwatson (MFC)
MFC after: 1 month
Because we now enforce UNIX98-style PTY's, we now use a lot of TTY's
that don't have the traditional /dev/ttyXX naming scheme. pkill(1)'s -t
flag automatically prepended the word "tty" to each TTY that was passed
on the command line. This meant that `pkill -t pts/0' was actually
converted to /dev/ttypts/0. Disable this broken behaviour for now.
Reported by: erwin
Initially, 'acl' (an 'acl_t *') is allocated, and its ACCESS_ACL and
DEFAULT_ACL fields are passed to the 'libc' ACL routines for subsequent
allocation. If the '-m' option (merge existing ACL with a new one) is
specified, then 'set_acl_mask()' will be called and passed one of the
two ACLs. This function, in turn, replaces this given ACL structure by
another, freshly allocated. However, the pointer in the 'acl' variable
in the caller is not updated. The caller then proceeds to free the ACL,
incurring in a double free condition.
Submitted by: Pedro Martelletto <pedro at ambientworks.net>
Approved by: rwatson (mentor)
for the convenience of rc.d. Now it has happily lived there for quite
a while. So move the pkill(1) source files from usr.bin to bin, too.
Approved by: gad
When I imported the MPSAFE TTY code, I added the -p flag to sh(1)'s
ulimit, but I forgot to document it in the appropriate manual page.
Requested by: stefanf
After the MPSAFE TTY import, we have support for the TAB0 and TAB3 flags
to handle tab expansion, while we only used to support OXTABS. Switch
stty(1) to use tab0 and tab3 to print whether tab expansion is turned on
or off. Implement the oxtabs and tabs switches by setting the
appropriate TABx value.
Even though POSIX only lists this as being XSI, we'd better follow it.
The last half year I've been working on a replacement TTY layer for the
FreeBSD kernel. The new TTY layer was designed to improve the following:
- Improved driver model:
The old TTY layer has a driver model that is not abstract enough to
make it friendly to use. A good example is the output path, where the
device drivers directly access the output buffers. This means that an
in-kernel PPP implementation must always convert network buffers into
TTY buffers.
If a PPP implementation would be built on top of the new TTY layer
(still needs a hooks layer, though), it would allow the PPP
implementation to directly hand the data to the TTY driver.
- Improved hotplugging:
With the old TTY layer, it isn't entirely safe to destroy TTY's from
the system. This implementation has a two-step destructing design,
where the driver first abandons the TTY. After all threads have left
the TTY, the TTY layer calls a routine in the driver, which can be
used to free resources (unit numbers, etc).
The pts(4) driver also implements this feature, which means
posix_openpt() will now return PTY's that are created on the fly.
- Improved performance:
One of the major improvements is the per-TTY mutex, which is expected
to improve scalability when compared to the old Giant locking.
Another change is the unbuffered copying to userspace, which is both
used on TTY device nodes and PTY masters.
Upgrading should be quite straightforward. Unlike previous versions,
existing kernel configuration files do not need to be changed, except
when they reference device drivers that are listed in UPDATING.
Obtained from: //depot/projects/mpsafetty/...
Approved by: philip (ex-mentor)
Discussed: on the lists, at BSDCan, at the DevSummit
Sponsored by: Snow B.V., the Netherlands
dcons(4) fixed by: kan
understand which code paths aren't possible.
This commit eliminates 117 false positive bug reports of the form
"allocate memory; error out if pointer is NULL; use pointer".
When I ported most applications away from <sgtty.h>, I noticed none of
them were actually using these definitions. I kept them in place,
because I didn't want to touch tools like pstat(8) and stty(1).
In preparation for the MPSAFE TTY layer, remove these definitions. This
doesn't have any impact with respect to binary compatibility (see
tty_conf.c).
We couldn now add an #error to <sys/ioctl_compat.h> when included
outside the kernel. Unfortunately, kdump's mkioctls includes this file
unconditionally.
Approved by: philip (mentor)
Cast string precision to `int'. amd64 systems warn about the
field precision being `long int' if we don't, and pathnames are
normally short enough to fit in an `int'.
Noticed by: pav
ln(1) checks to see if the source of a symlink, i.e. the file it
should point to actually exists. The default is the old ln
behavior, that does not check, to avoid surprising people who may
be using ln(1) in scripts or other non-interactive places.
PR: bin/7265
Submitted by: Joel Ray Holveck, detlev!joelh at mail.camalott.com
MFC after: 2 weeks
Portability Utilities" option.
Often configure scripts generated by the autotools test if $LINENO works and
refuse to use /bin/sh if not.
Package test run by: pav
