Commit Graph

68 Commits

Author SHA1 Message Date
jilles
c86bc993b1 sh: Improve comments in expand.c. 2010-09-05 21:12:48 +00:00
jilles
44306000bd sh: Remove remnants of '!!' to negate pattern.
This Almquist extension was disabled long ago.

In pathname generation, components starting with '!!' were treated as
containing wildcards, causing unnecessary readdir (which could fail, causing
pathname generation to fail while it should not).
2010-08-22 21:18:21 +00:00
jilles
8824c5ab76 sh: Fix heap-based buffer overflow in pathname generation.
The buffer for generated pathnames could be too small in some cases. It
happened to be always at least PATH_MAX long, so there was never an overflow
if the resulting pathnames would be usable.

This bug may be abused if a script subjects input from an untrusted source
to pathname generation, which a bad idea anyhow. Most shell scripts do not
work on untrusted data. secteam@ says no advisory is necessary.

PR:		bin/148733
Reported by:	Changming Sun snnn119 at gmail com
MFC after:	10 days
2010-08-10 22:45:59 +00:00
jilles
8fcbe1caf8 sh: Forget about terminated background processes sooner.
Unless $! has been referenced for a particular job or $! still contains that
job's pid, forget about it after it has terminated. If $! has been
referenced, remember the job until the wait builtin has reported its
completion (either with the pid as parameter or without parameters).

In interactive mode, jobs are forgotten after termination has been reported,
which happens before primary prompts and through the jobs builtin. Even
then, though, remember a job if $! has been referenced.

This is similar to what is suggested by POSIX and should fix most memory
leaks (which also tend to cause sh to use more CPU time) with long running
scripts that start background jobs.

Caveats:
* Repeatedly referencing $! without ever doing 'wait', like
    while :; do foo & echo started foo: $!; sleep 60; done
  will still use a lot of memory and CPU time in the long run.
* The jobs and jobid builtins do not cause a job to be remembered for longer
  like expanding $! does.

PR:		bin/55346
2010-06-29 22:37:45 +00:00
jilles
c49fe4933f sh: Fix pathname expansion with quoted slashes like *\/.
These are git commits 36f0fa8fcbc8c7b2b194addd29100fb40e73e4e9 and
d6d06ff5c2ea0fa44becc5ef4340e5f2f15073e4 in dash.

Because this is the first code I'm importing from dash to expand.c, add the
Herbert Xu copyright notice which is in dash's expand.c.

When pathname expanding *\/, the CTLESC representing the quoted state was
erroneously taken as part of the * pathname component. This CTLESC was then
seen by the pattern matching code as escaping the '\0' terminating the
string.

The code is slightly different because dash converts the CTLESC characters
to backslashes and removes all the other CTL* characters to allow
substituting glob(3).

The effect of the bug was also slightly different from dash (where nothing
matched at all). Because a CTLESC can escape a '\0' in some way, whether
files were included despite the bug depended on memory that should not be
read. In particular, on many machines /*\/ expanded to a strict subset of
what /*/ expanded to.

Example:
  echo /*"/null"

This should print /dev/null, not /*/null.

PR:		bin/146378
Obtained from:	dash
2010-05-11 23:19:28 +00:00
jilles
88403fad18 sh: Use stalloc for arith variable names.
This is simpler than the custom memory tracker I added earlier, and is also
needed by the dash arith code I plan to import.
2010-04-25 20:43:19 +00:00
jilles
d21d692410 sh: Do tilde expansion in substitutions.
This applies to word in ${v-word}, ${v+word}, ${v=word}, ${v?word} (which
inherits quoting from the outside) and in ${v%word}, ${v%%word}, ${v#word},
${v##word} (which does not inherit any quoting).

In all cases tilde expansion is only attempted at the start of word, even if
word contains spaces. This agrees with POSIX and other shells.

This is the last part of the patch tested in the exp-run.

Exp-run done by: erwin (with some other sh(1) changes)
2010-04-03 22:04:44 +00:00
jilles
8b7edeca44 sh: Allow quoting pattern match characters in ${v%pat} and ${v#pat}.
Note that this depends on r206145 for allowing pattern match characters to
have their special meaning inside a double-quoted expansion like "${v%pat}".

PR:		bin/117748
Exp-run done by:	erwin (with some other sh(1) changes)
2010-04-03 21:07:50 +00:00
jilles
446838eef9 sh: Fix some bugs with backquoted builtins:
- correctly handle error output in $(builtin 2>&1), clarify out1/out2 vs
  output/errout in the code
- treat all builtins as regular builtins so errors do not abort the shell
  and variable assignments do not persist
- respect the caller's INTOFF

Some bugs still exist:
- expansion errors may still abort the shell
- some side effects of expansions and builtins persist
2010-01-01 18:17:46 +00:00
jilles
5e8a2136e7 sh: Various warning fixes (from WARNS=6 NO_WERROR=1):
- const
- initializations to silence -Wuninitialized (it was safe anyway)
- remove nested extern declarations
- rename "index" locals to "idx"
2009-12-27 18:04:05 +00:00
jilles
d5d0bbbb70 sh: Do not consider a tilde-prefix with expansions in it.
That is, do not do tilde expansion if any of the CTL* bytes (\201-\210), not
only CTLESC and CTLQUOTEMARK, are encountered. Such an expansion would look
up a user name with sh's internal representation.

The parser does not currently distinguish between backslashed and
unbackslashed \201-\210, so tilde expansion of user names with these bytes
in them is not so easy to fix.
2009-12-25 15:29:18 +00:00
jilles
8e458de852 sh: Constify various strings.
Most of this is adding const keywords, but setvar() in var.c had to be
changed somewhat more.
2009-12-24 18:41:14 +00:00
jilles
ae4c008ec3 sh: Exempt $@ and $* from set -u
This seems more useful and will likely be in the next POSIX standard.

Also document more precisely in the man page what set -u does (note that
$@, $* and $! are the only special parameters that can ever be unset, all
the others are always set, although they may be empty).
2009-10-24 21:20:04 +00:00
jilles
7dc0587c35 Fix some weirdnesses in the NetBSD IFS code,
in particular "$@"$ifschar if the final positional parameter is empty.
With the NetBSD code, adding the $ifschar removes a parameter.

PR:		standards/79067
Approved by:	ed (mentor) (implicit)
2009-06-25 17:14:06 +00:00
jilles
a02858661b Improve IFS expansion using code from NetBSD.
We now pass the ifs.sh testsuite.

PR:		standards/79067
Approved by:	ed (mentor) (implicit)
Obtained from:	NetBSD
2009-06-25 17:10:51 +00:00
stefanf
91768cb1d9 Expand $LINENO to the current line number. This is required by SUSv3's "User
Portability Utilities" option.

Often configure scripts generated by the autotools test if $LINENO works and
refuse to use /bin/sh if not.

Package test run by:	pav
2008-05-15 19:55:27 +00:00
stefanf
44778d89a5 Sigh, when reapplying the patch to HEAD, I somehow forgot to commit this file.
Reported by:	Jaakko Heinonen
2008-04-28 07:26:34 +00:00
stefanf
085fc40080 Fix expanding of quoted positional parameters in case patterns.
Obtained from:	NetBSD (expand.c 1.58 and 1.59)
Submitted by:	Paul Jarc
PR:		56147
2006-11-07 22:46:13 +00:00
stefanf
d02f26394e When parsing an invalid parameter expansion (eg. ${} or ${foo@bar}) do not
issue a syntax error immediately but save the information that it is erroneous
for later when the parameter expansion is actually done.  This means eg. "false
&& ${}" will not generate an error which seems to be required by POSIX.
Include the invalid parameter expansion in the error message (sometimes
abbreviated with ... because recovering it would require a lot of code).

PR:		105078
Submitted by:	emaste
2006-11-05 18:36:05 +00:00
schweikh
627562241f Remove some white space at EOL. 2006-02-04 14:37:50 +00:00
rse
fe548cd4fa Various small code cleanups resulting from a code reviewing
and linting procedure:

1. Remove useless sub-expression:

   - if (*start || (!ifsspc && start > string && (nulonly || 1))) {
   + if (*start || (!ifsspc && start > string)) {

   The sub-expression "(nulonly || 1)" always evaluates to true and
   according to CVS logs seems to be just a left-over from some
   debugging and introduced by accident. Removing the sub-expression
   doesn't change semantics and a code inspection showed that the
   variable "nulonly" is also not necessary here in any way (and the
   expression would require fixing instead of removing).

2. Remove dead code:

   -                if (backslash && c == '\\') {
   -                        if (read(STDIN_FILENO, &c, 1) != 1) {
   -                                status = 1;
   -                                break;
   -                        }
   -                        STPUTC(c, p);
   -                } else if (ap[1] != NULL && strchr(ifs, c) != NULL) {
   +                if (ap[1] != NULL && strchr(ifs, c) != NULL) {

   Inspection of the control and data flow showed that variable
   "backslash" is always false (0) when the "if"-expression is
   evaluated, hence the whole block is effectively dead code.
   Additionally, the skipping of characters after a backslash is already
   performed correctly a few lines above, so this code is also not
   needed at all. According to the CVS logs and the ASH 0.2 sources,
   this code existed in this way already since its early days.

3. Cleanup Style:

   - ! trap[signo][0] == '\0' &&
   + ! (trap[signo][0] == '\0') &&

   The expression wants to ensure the trap is not assigned the empty
   string. But the "!" operator has higher precedence than "==", so the
   comparison should be put into parenthesis to form the intended way of
   expression. Nevertheless the code was effectively not really broken
   as both particular NUL comparisons are semantically equal, of course.
   But the parenthesized version is a lot more intuitive.

4. Remove shadowing variable declaration:

   - char *q;

   The declaration of symbol "q" hides another identical declaration of
   "q" in the same context. As the other "q" is already reused multiple
   times and also can be reused again without negative side-effects,
   just remove the shadowing declaration.

5. Just small cosmetics:

   - if (ifsset() != 0)
   + if (ifsset())

   The ifsset() macro is already coded by returning the boolean result
   of a comparison operator, so no need to compare this boolean result
   again against a numerical value. This also aligns the macros usage to
   the remaining existing code.

Reviewed by: stefanf@
2005-09-06 19:30:00 +00:00
delphij
909e7ff886 Fix a bug when shell expansion is done against dangling symlinks, by
converting the stat() call to a lstat() call, which will cover the
situation.  One can exercise this bug by referring a dangling link with
something like */the-link.

Approved by:	re (scottl)
Submitted by:	Simon 'corecode' Schubert [corecode fs ei tum de]
Obtained from:	NetBSD via DragonFlyBSD (NetBSD rev. 1.51 and DragonFly
		rev. 1.6)
MFC After:	3 days
2005-07-07 18:10:33 +00:00
markm
4383f14801 Remove clause 3 from the UCB licenses.
OK'ed by:	imp, core
2004-04-06 20:06:54 +00:00
ache
df9eb04e40 Remove collate_range_cmp() stabilization, it conflicts with ranges 2003-08-03 04:28:10 +00:00
dds
34719e1dbb Changes following CScout analysis:
- Removed dead declarations
- Made objects that should have been declared as static, static.

The changes use STATIC instead of static, following the existing
convention in the rest of the code.

Approved by:	schweikh (mentor)
MFC after:	2 weeks
2003-07-05 15:18:44 +00:00
fenner
99d6402b40 Instead of eating trailing newlines after inserting them into the
output buffer, don't insert them at all.  This prevents a buffer
*underrun* when the substitution consists completely of newlines
(e.g. `echo`) and the byte before the source buffer to which p
points is a '\n', in which case more characters would be removed
from the output buffer than were inserted.

This fixes certain port builds on sparc64.

Approved by:	re (scottl)
Reviewed by:	des, tjr
2003-05-31 06:27:57 +00:00
ru
6999b65340 Fixed (soon might be fatal) -Wformat warnings. 2003-03-15 07:56:59 +00:00
tjr
d53165c37b Do not strip CTL* escapes from redirection filenames in exptilde(),
expari(), expbackq() and evalvar(). Similar to revision 1.39.
Patch from Tor Egge.

PR:		45349
MFC after:	2 weeks
2003-01-08 10:50:08 +00:00
tjr
3b9687df3d Add the "wordexp" shell built-in command which will be used to implement
the POSIX wordexp() function.
2002-12-26 14:28:54 +00:00
tjr
33d1a8b9c2 Do not strip CTL* escapes from redirection filenames in argstr(); they
are later stripped with rmescapes() in expandarg(). If the filename has
already been unescaped, doing it again in rmescapes() can walk off the
end of the string, leading to memory corruption and eventually SIGSEGV.

Noticed by:	kris
2002-10-08 11:22:49 +00:00
charnier
c88222c6c0 Replace various spellings with FALLTHROUGH which is lint()able 2002-08-25 13:01:47 +00:00
obrien
f6f518b207 Consistently use FBSDID 2002-06-30 05:15:05 +00:00
tjr
b19972e699 Implement the -u (-o nounset) option, which gives an error message if
an unset variable is expanded.

Obtained from:	NetBSD (bjh21, christos)
2002-05-19 08:30:16 +00:00
imp
5ef5088ac4 o __P has been reoved
o Old-style K&R declarations have been converted to new C89 style
o register has been removed
o prototype for main() has been removed (gcc3 makes it an error)
o int main(int argc, char *argv[]) is the preferred main definition.
o Attempt to not break style(9) conformance for declarations more than
  they already are.
o Change
	int
	foo() {
	...
  to
	int
	foo(void)
	{
	...
2002-02-02 06:50:57 +00:00
tegge
b0e7279ba2 Don't check uninitialized memory for having the shell control character
value CTLARI since this might break expansion of arithmetic expressions.

Don't access memory below start of stackblock.

Problem analyzed by hunt@iprg.nokia.com, slightly different patch applied.

PR:		24443
Submitted by:	hunt@iprg.nokia.com
2001-09-19 20:16:38 +00:00
tegge
0c39138a0b BASESYNTAX, DQSYNTAX, SQSYNTAX and ARISYNTAX handles negative indexes.
Allow those to be used to properly quote characters in the shell
control character range.
2001-09-19 20:07:47 +00:00
cracauer
31c479dcd8 Fix environment passung to eval'ed commands.
PR:		bin/6577
Submitted by:	Anatoly Vorobey <mellon@pobox.com>
Approved by:	silence amoung other sh experts
2000-05-15 12:33:17 +00:00
cracauer
ea70a76b31 First round of 8-bit fixes. 1999-12-15 11:46:32 +00:00
cracauer
46e40e0c76 Fix "subscript has type `char'" warnings by casting to int, as
discussed on -arch.
1999-12-04 17:12:47 +00:00
peter
66312e4a8d $Id$ -> $FreeBSD$ 1999-08-27 23:15:48 +00:00
kris
713dd62834 Various spelling/formatting changes.
Submitted by: Philippe Charnier <charnier@xp11.frmug.org>
1999-05-08 10:22:15 +00:00
cracauer
da2b842fad Next approach to make loops in interactive interruptable.
PR:		bin/9173
1999-04-21 11:52:39 +00:00
tegge
82a8c819f3 During variable expansion, the internal representation of the expression
might be relocated.  Handle this case.
PR:		7059
1999-04-13 04:13:09 +00:00
tegge
e35119550e When a variable expansion is enclosed in double quotes, the internal
representation of the expression is quoted.  Take care of this when
doing pattern matching in conjunction with trimming.

	#!/bin/sh
	c=d:e; echo "${c%:e}"

PR:		NetBSD PR#7231
Noticed by:	Havard Eidnes <Havard.Eidnes@runit.sintef.no>
1999-04-09 15:23:48 +00:00
tegge
c82e121ce7 Be more consistent with handling of quote mark control character.
Don't output double-quotes inside variable expansion/arithmetic
expansion region in here-documents.  When leaving the arithmetic
expansion syntax mode, adjust the dblquote flag according to
previous syntax, in order to avoid splitting of quoted variables.
1998-09-13 19:24:57 +00:00
tegge
ac59a0c5cd Better handling of word splitting. Don't record the same region
multiple times when performing nested variable expansion, and
preserve some quoting information in order to avoid removing
apparently empty expansion result.
1998-09-06 21:13:09 +00:00
charnier
86bc037e71 Add rcsid. Spelling. 1998-05-18 06:44:24 +00:00
jkh
e5f80c5114 Back out previous fix - this bug's got diplomatic immunity as a registered
political issue.
1997-06-19 17:57:06 +00:00
jkh
7f875c5f66 >Number: 3780
>Category:       bin
>Synopsis:       WEXITSTATUS() may return nagative value, which causes sh to generate bad $?

PR:		3780
Submitted by:	sanewo@ba2.so-net.or.jp
1997-06-19 15:14:01 +00:00
ache
837346c6cf Now [^abc] means the same as [!abc] like bash and *csh already does 1997-06-06 23:04:33 +00:00