Commit Graph

2985 Commits

Author SHA1 Message Date
Jilles Tjoelker
72238faa6a sh: Remove arbitrary length limit on << EOF markers.
This also simplifies the code.
2014-09-14 16:46:30 +00:00
Jilles Tjoelker
671a890e58 sh: Make checkend() a real function instead of an emulated nested function.
No functional change is intended, but the generated code is slightly
different.
2014-09-14 16:27:49 +00:00
Jilles Tjoelker
5545fadd84 sh: Add some const keywords. 2014-09-14 15:59:15 +00:00
Jilles Tjoelker
cd60e2c67d sh: Allow enabling job control without a tty in non-interactive mode.
If no tty is available, 'set -m' is still useful to put jobs in their own
process groups.
2014-09-04 21:48:33 +00:00
Warner Losh
a83c9ec1e5 Update the date for last example.
Sponsored by: Netflix
2014-08-30 02:12:58 +00:00
Warner Losh
56fd128433 Add canonical population of a disk / thumb drive from an image
example.
2014-08-28 21:30:39 +00:00
Mateusz Guzik
8b04bbef31 Return real parent pid in kinfo (used by e.g. ps)
Add a separate field which exports tracer pid and add a new keyword
("tracer") for ps to display it.

This is a follow up to r270444.

Reviewed by:	kib
MFC after:	1 week
Relnotes:	yes
2014-08-28 08:41:11 +00:00
Pedro F. Giffuni
0fa46a4231 Always check the limits of array index variables before using them.
Obtained from:	DragonFlyBSD
MFC after:	1 week
2014-08-21 02:40:33 +00:00
Jilles Tjoelker
dd6d480a3e sh: Remove two redundant (uintmax_t) casts.
Submitted by:	jmallett
2014-08-20 20:15:43 +00:00
Bryan Drewery
5608fd23c2 Revert r267233 for now. PIE support needs to be reworked.
1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
   build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
   where it never would work anyhow, such as csu or loader. This suggests
   there may be better ways of adding support to the tree. Many of these
   cases can be fixed such that -fPIE will work but there is really no
   reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
   really building libraries but have been using bsd.prog.mk because the code
   is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
   been needed.

We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.

Reported by:	kib
2014-08-19 15:04:32 +00:00
Jilles Tjoelker
ef9e61785a sh: Avoid overflow in atoi() when parsing HISTSIZE.
Side effect: a non-numeric HISTSIZE now results in the default size (100)
instead of 0.
2014-08-17 19:36:56 +00:00
Jilles Tjoelker
d53f7f64f7 sh: Reject integer overflow in number and is_number. 2014-08-17 16:40:29 +00:00
Jilles Tjoelker
1c9c6ea481 sh: Don't hardcode relative paths in the tests stderr files.
These paths have had to be adjusted to changes in the testsuite runner
several times, so modify the tests to remove the need for such adjustment.

A cp in functional_test.sh is now unneeded, but this matters little in
performance.
2014-08-17 14:26:12 +00:00
Jilles Tjoelker
d5b14891eb sh: Mask off shift distance (<< and >>) in arithmetic.
In C, shift distances equal to or larger than the number of bits in the
operand result in undefined behaviour. As part of eliminating undefined
behaviour in arithmetic, mask off the distance like Java and JavaScript
specify and C on x86 usually does.

Assumption: conversion from unsigned to signed retains the two's complement
bits.
Assumption: uintmax_t has no padding bits.
2014-08-15 22:36:41 +00:00
Alan Somers
da02ed7331 Skip pgrep-j and pkill-j if jail or jls is not installed.
Even though jail is part of the base system, it can be disabled by src.conf
settings. Therefore, it should be listed as a required program for tests
that use it.

CR:		D603
MFC after:	3 days
Sponsored by:	Spectra Logic
2014-08-14 14:59:40 +00:00
Enji Cooper
9f31240773 Port date/bin/tests to ATF
Phabric: D545
Approved by: jmmv (mentor)
Submitted by: keramida (earlier version)
MFC after: 2 weeks
Sponsored by: Google, Inc
Sponsored by: EMC / Isilon Storage Division
2014-08-13 04:43:29 +00:00
Enji Cooper
12e2df3c36 Convert bin/sh/tests to ATF
The new code uses a "test discovery mechanism" to determine
what tests are available for execution

The test shell can be specified via:

  kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh

Sponsored by: EMC / Isilon Storage Division
Approved by: jmmv (mentor)
Reviewed by: jilles (maintainer)
2014-08-13 04:14:50 +00:00
Konstantin Belousov
d735998057 Correct the problems with the ptrace(2) making the debuggee an orphan.
One problem is inferior(9) looping due to the process tree becoming a
graph instead of tree if the parent is traced by child. Another issue
is due to the use of p_oppid to restore the original parent/child
relationship, because real parent could already exited and its pid
reused (noted by mjg).

Add the function proc_realparent(9), which calculates the parent for
given process. It uses the flag P_TREE_FIRST_ORPHAN to detect the head
element of the p_orphan list and than stepping back to its container
to find the parent process. If the parent has already exited, the
init(8) is returned.

Move the P_ORPHAN and the new helper flag from the p_flag* to new
p_treeflag field of struct proc, which is protected by proctree lock
instead of proc lock, since the orphans relationship is managed under
the proctree_lock already.

The remaining uses of p_oppid in ptrace(PT_DETACH) and process
reapping are replaced by proc_realparent(9).

Phabric:	D417
Reviewed by:	jhb
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2014-08-07 05:47:53 +00:00
Jilles Tjoelker
4d34663be3 sh: Allow arbitrarily large numbers in break and continue.
The argument is capped to loopnest, so strtol()'s [ERANGE] can be ignored.
2014-07-20 20:29:09 +00:00
Jilles Tjoelker
e61ae4ffc8 sh: Remove prefix() function. Use strncmp() instead. 2014-07-20 12:06:52 +00:00
Jilles Tjoelker
2ef146f14c sh: Deduplicate some code in ulimit builtin. 2014-07-19 14:06:23 +00:00
Brooks Davis
80189b3b09 Replace all uses of libncurses and libtermcap with their wide character
variants.  This allows usable file system images (i.e. those with both a
shell and an editor) to be created with only one copy of the curses library.

Exp-run:	antoine
PR:		189842
Discussed with:	bapt
Sponsored by:	DARPA, AFRL
2014-07-17 18:24:34 +00:00
Jilles Tjoelker
7b9104c0a9 sh: Correctly handle positional parameters beyond INT_MAX on 64-bit systems.
Currently, there can be no more than INT_MAX positional parameters. Make
sure to treat all higher ones as unset to avoid incorrect results and
crashes.

On 64-bit systems, our atoi() takes the low 32 bits of the strtol() and
sign-extends them.

On 32-bit systems, the call to atoi() returned INT_MAX for too high values
and there is not enough address space for so many positional parameters, so
there was no issue.
2014-07-12 21:54:11 +00:00
Jilles Tjoelker
5ddabb8348 sh: Consistently treat ${01} like $1.
Leading zeroes were ignored when checking whether a positional parameter is
set, but not when expanding its value. Ignore leading zeroes in any case.
2014-07-12 10:27:30 +00:00
Jilles Tjoelker
f1058ba09a sh: Add test for ${01} and ${010} that already works.
Although it is probably unwise to use this, POSIX is clear that leading
zeroes are permitted in positional parameters (and do not indicate octal).

Such positional parameters are checked for being unset and/or null
correctly, but their value is incorrectly expanded.
2014-07-08 22:04:44 +00:00
Jilles Tjoelker
725d072732 Don't install locale1.0 if MK_NLS == no.
The test locale1.0 depends on locale support; it is meaningless without a
working LC_MESSAGES.

I added an OptionalObsoleteFiles.inc entry.

PR:		181151
Submitted by:	Garrett Cooper (original version)
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-07-08 21:50:13 +00:00
Warner Losh
413a368c90 rm -rf can fail sometimes with an error from fts_read. Make it honor
fflag to ignore fts_read errors, but stop deleting from that directory
because no further progress can be made.

When building a kernel with a high -j value on a high core count
machine, during the cleanobj phase we can wind up doing multiple rm
-rf at the same time for modules that have subdirectories. This
exposed this race (sometimes) as fts_read can return an error if the
directory is removed by another rm -rf. Since the intent of the -f
flag was to ignore errors, even if this was a bug in fts_read, we
should ignore the error like we've been instructed to do.
2014-07-07 23:21:20 +00:00
Jilles Tjoelker
2d70c63720 sh: Fix overflow checking on 'ulimit' operand. 2014-07-05 21:50:59 +00:00
Xin LI
c15455feb3 Check if fchflags() is needed by fstat'ing before and check
the results.

Reviewed by:	jilles
X-MFC-With:	r267977
2014-07-01 22:46:39 +00:00
Xin LI
e6683f19dd Always set UF_ARCHIVE on target (because they are by definition new files
and should be archived) and ignore error when we can't set it (e.g. NFS).

Reviewed by:	ken
MFC after:	2 weeks
2014-06-27 19:57:54 +00:00
Baptiste Daroussin
bd0891ceb3 use .Mt to mark up email addresses consistently (part1)
PR:		191174
Submitted by:	Franco Fichtner  <franco@lastsummer.de>
2014-06-20 09:40:43 +00:00
Joel Dahl
cf20d40493 mdoc: remove superfluous paragraph macros. 2014-06-15 10:43:12 +00:00
Bryan Drewery
864c53ead8 In preparation for ASLR [1] support add WITH_PIE to support building with -fPIE.
This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.

Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]

Mark known build failures as NO_PIE for now.

The only known runtime failure was rtld.

[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by:		Shawn Webb <lattera@gmail.com>
Discussed between:	des@ and Shawn Webb [2]
2014-06-08 17:29:31 +00:00
Sergey Kandaurov
38d22923f7 Add support for inspecting process flags set in p_flag2.
Reviewed by:	jhb
MFC after:	1 week
2014-06-06 22:52:11 +00:00
Warner Losh
04efeffe99 When building picobsd, define WITHOUT_OPENSSL and WITHOUT_KERBEROS and
remove the now-redundant checks for RELEASE_CRUNCH. This originally
was defined for building smaller sysinstall images, but was later also
used by picobsd builds for a similar purpose. Now that we've moved
away from sysinstall, picobsd is the only remaining consumer of this
interface. Adding these two options reduces the RELEASE_CRUNCH
special cases in the tree by half.
2014-06-06 04:09:07 +00:00
Jilles Tjoelker
b0762e4921 sh: Avoid undefined behaviour shifting negative values left in arithmetic.
With i386 base clang, arith_yacc.o remains unchanged.
2014-06-01 11:25:34 +00:00
Jilles Tjoelker
2ceda70233 sh: Simplify find_command() slightly. 2014-05-31 22:25:45 +00:00
Jilles Tjoelker
c8fb3e69d0 sh: In getopts, unset OPTARG where POSIX says we should. 2014-05-10 19:18:49 +00:00
Jilles Tjoelker
8207fd5f81 sh: Add new tests to the Makefile. 2014-05-10 19:06:36 +00:00
Jilles Tjoelker
6e76445cf7 sh: Don't discard getopts state on unknown option or missing argument.
When getopts finds an invalid option or a missing option-argument, it should
not reset its state and should set OPTIND as normal. This is an old ash bug
that was fixed long ago in dash. Our behaviour now matches most other
shells.
2014-05-10 17:42:21 +00:00
Jilles Tjoelker
8c4e5fc061 sh: Send getopts error messages to stderr, not stdout.
Adjust a testcase for this change.
2014-05-09 13:32:36 +00:00
Jilles Tjoelker
bc7f6652dd sh: Add more necessary INTOFF/INTON. 2014-05-09 13:27:30 +00:00
Alan Somers
540c78258c Incorporate feedback from bde and jilles regarding r265472 to dd(1).
* Don't use sysexits.h.  Just exit 1 on error and 0 otherwise.
* Don't sacrifice precision by converting the output of clock_gettime() to a
  double and then comparing the results.  Instead, subtract the values of
  the two clock_gettime() calls, then convert to double.
* Don't use CLOCK_MONOTONIC_PRECISE.  It's an unportable synonym for
  CLOCK_MONOTONIC.
* Use more appropriate names for some local variables.
* In the summary message, round elapsed time to the nearest microsecond.

Reported by:	bde, jilles
MFC after:	3 days
X-MFC-With:	265472
2014-05-08 19:10:04 +00:00
Jilles Tjoelker
5699ef2588 sh: Add some tests for normal use of getopts. 2014-05-07 21:45:25 +00:00
Thomas Quinot
aa7a161b40 (dd_out): Fix handling of all-zeroes block at end of input with
conv=sparse.

This change fixes two separate issues observed when the last output
block is all zeroes, and conv=sparse is in use. In this case, care
must be taken to roll back the last seek and write the entire last zero
block at the original offset where it should have occurred: when the
destination file is a block device, it is not possible to roll back
by just one character as the write would then not be properly aligned.

Furthermore, the buffer used to write this last all-zeroes block
needs to be properly zeroed-out. This was not the case previously,
resulting in a junk data byte appearing instead of a zero in the
output stream.

PR:		bin/189174
PR:		bin/189284
Reviewed by:	kib
MFC after:	2 weeks
2014-05-07 19:33:29 +00:00
Alan Somers
d1d66eac15 dd(1) uses gettimeofday(2) to compute the throughput statistics. However,
gettimeofday returns the system clock, which may jump forward or back,
especially if NTP is in use.  If the time jumps backwards, then dd will see
negative elapsed time, round it up to 1usec, and print an absurdly fast
transfer rate.

The solution is to use clock_gettime(2) with CLOCK_MONOTONIC_PRECISE as the
clock_id.  That clock advances steadily, regardless of changes to the system
clock.

Reviewed by:	delphij
MFC after:	3 days
Sponsored by:	Spectra Logic
2014-05-06 22:06:39 +00:00
Warner Losh
c6063d0da8 Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.
2014-05-06 04:22:01 +00:00
Thomas Quinot
37f1b88ba0 Minor comment fix: dbsz is the block size as set by bs/ibs/obs,
not the size of the buffer.
2014-05-03 09:07:02 +00:00
Thomas Quinot
66d5f71966 Minor comment fix. 2014-05-03 09:06:11 +00:00
Bryan Drewery
13767130c7 Add -J to filter by matching jail IDs and names.
-J 0 can be used to show only host processes.

Patch partially based on work by bz@

PR:		bin/78763
MFC after:	2 weeks
Relnotes:	yes
2014-05-02 15:05:47 +00:00