Commit Graph

2905 Commits

Author SHA1 Message Date
gjb
5f63f50572 MFC r268376 (imp):
rm -rf can fail sometimes with an error from fts_read. Make it
  honor fflag to ignore fts_read errors, but stop deleting from
  that directory because no further progress can be made.

  When building a kernel with a high -j value on a high core count
  machine, during the cleanobj phase we can wind up doing multiple
  rm -rf at the same time for modules that have subdirectories. This
  exposed this race (sometimes) as fts_read can return an error if
  the directory is removed by another rm -rf. Since the intent of
  the -f flag was to ignore errors, even if this was a bug in
  fts_read, we should ignore the error like we've been instructed
  to do.

Approved by:	re (kib)
Sponsored by:	The FreeBSD Foundation
2014-10-01 16:18:40 +00:00
ngie
0d7257f533 MFC r269903:
Port date/bin/tests to ATF

 Phabric: D545
 Approved by: jmmv (mentor)
 Submitted by: keramida (earlier version)
 Sponsored by: Google, Inc
 Sponsored by: EMC / Isilon Storage Division
2014-08-29 02:21:02 +00:00
pfg
564b330f17 MFC r270256:
Always check the limits of array index variables before using them.

Obtained from:	DragonFlyBSD
2014-08-28 18:11:05 +00:00
kib
afeea342d6 MFC r269656:
Implement and use proc_realparent(9).

MFC r270024 (by markj):
Correct the order of arguments passed to LIST_INSERT_AFTER().

For merge, the p_treeflag member of struct proc was moved to the end
of the structure, to keep KBI intact.
2014-08-21 10:46:19 +00:00
jilles
5d7615994f MFC r268429: Don't install locale1.0 if MK_NLS == no.
The test locale1.0 depends on locale support; it is meaningless without a
working LC_MESSAGES.

I added an OptionalObsoleteFiles.inc entry.

PR:		181151
Submitted by:	Garrett Cooper (original version)
Sponsored by:	EMC / Isilon Storage Division
2014-08-20 19:39:07 +00:00
asomers
5482ed4923 MFC r269977
Skip pgrep-j and pkill-j if jail or jls is not installed.

Even though jail is part of the base system, it can be disabled by src.conf
settings. Therefore, it should be listed as a required program for tests that
use it.
2014-08-18 20:21:12 +00:00
pluknet
42ff249abc MFC r267196:
Add support for inspecting process flags set in p_flag2.
2014-06-26 08:55:06 +00:00
thomas
e767419662 MFC rev. 265593:
(dd_out): Fix handling of all-zeroes block at end of input with
conv=sparse.

PR:		bin/189174
PR:		bin/189284
Reviewed by:	kib
2014-05-21 07:21:36 +00:00
bdrewery
a0f7dc4feb MFC r265229,r265239:
Add -J to filter by matching jail IDs and names.
2014-05-17 02:32:47 +00:00
dumbbell
ee189cb17a date(1): Add "-R" flag to use RFC 2822 date and time output format [MFC]
As stated in the man page, this is equivalent to use "%a, %d %b %Y %T %z"
as the output format while LC_TIME is set to the "C" locale.

This is compatible with date(1) from the GNU core utilities.

This is an MFC of r264968 and r264970.
2014-05-03 13:19:11 +00:00
jmmv
1e4589b921 MFC various moves of tools/regressions/ tests to the new infrastructure.
- r263220 Migrate tools/regression/sbin/ to the new tests layout.
- r263222 Add Makefile missed in r263220.
- r263226 Migrate tools/regression/{usr.bin/lastcomm,usr.sbin}/ to the new tests layout.
- r263227 Migrate most of tools/regression/usr.bin/ to the new tests layout.
- r263345 Expand tabs that sneaked in into spaces.
- r263346 Migrate tools/regression/usr.bin/make/ to the new tests layout.
- r263348 Add Makefiles missed in r263346.
- r263351 Migrate tools/regression/usr.bin/pkill/ to the new tests layout.
- r263388 Mark multi_test as requiring /usr/share/dict/words.
- r263814 Fix path to the run.pl script to let these tests run.
- r264742 Prevent building tests when bootstrapping make.

This is 'make tinderbox' clean.
2014-04-27 01:15:10 +00:00
delphij
4882a37127 MFC r264059+264067:
Implement GNU's extension of 'status' operand.  The GNU syntax is
borrowed where syntax status=noxfer means no transfer statistics
and status=none means no status information at all.

This feature is useful because the statistics information can
sometimes be annoying, and redirecting stderr to /dev/null would
mean error messages also gets silenced.

Obtained from:	OpenBSD
2014-04-17 00:31:20 +00:00
jilles
07966b660e MFC r263847: sh: Fix memory leak with an assignment before a regular builtin 2014-04-15 18:29:05 +00:00
jilles
7ba0eb2fc1 MFC r263846: sh: Fix memory leak when trying to set a read only variable. 2014-04-15 18:24:24 +00:00
jmmv
63fa6c9a5b MFC refactoring of the *.test.mk files.
- r263161 Make bsd.test.mk the only public mk fragment for the building of tests.
- r263172 Move FreeBSD Test Suite-specific code to a suite.test.mk file.
- r263204 Add some documentation for bsd.test.mk.
- r263217 Document support for TAP-compliant Perl test programs.

This is "make tinderbox" clean.
2014-04-14 23:51:57 +00:00
jilles
813864aadb MFC r263777: sh: Fix possible memory leaks and double frees with unexpected
SIGINT.
2014-04-14 21:26:34 +00:00
dim
b4a5074609 MFC r263778:
Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process
all the SUBDIR entries in parallel, instead of serially.  Apply this
option to a selected number of Makefiles, which can greatly speed up the
build on multi-core machines, when using make -j.

This can be extended to more Makefiles later on, whenever they are
verified to work correctly with parallel building.

I tested this on a 24-core machine, with make -j48 buildworld (N = 6):

		before    stddev       after    stddev
		=======   ======       =======  ======
real time        1741.1     16.5         959.8     2.7
user time       12468.7     16.4       14393.0    16.8
sys  time        1825.0     54.8        2110.6    22.8

(user+sys)/real     8.2                   17.1

E.g. the build was approximately 45% faster in real time.  On machines
with less cores, or with lower -j settings, the speedup will not be as
impressive.  But at least you can now almost max out a machine with
buildworld!

Submitted by:	jilles

MFC r263833:

Enable parallel building for gnu/usr.bin and usr.bin/clang too.
2014-04-09 18:16:58 +00:00
jilles
b37f0c842b MFC r263195: sh: Add some consts. 2014-04-05 20:01:08 +00:00
jilles
76e2b229a4 MFC r262755: sh: Make argstr() return where it stopped and simplify expari()
using this.
2014-04-05 19:43:47 +00:00
jmmv
80efbcbbcd Sync sh(1) in stable/10 to head.
This is a MFC of all the commits listed below.

My original goal of this change was to only merge the move of the tests
from tools/regression/bin/ into the new layout (which include tests for
sh(1) and other tools as well).  However, doing so is tricky due to the
ongoing work in sh(1) and, especially, the many changes to its tests
since stable/10 was first branched.

Merging everything is the simplest way to achieve this goal and, as a
bonus point, we get various fixes and miscellaneous improvements into
the branch.

Per jilles' suggestion, I'm avoiding the merge of a couple of changes
(r256850 and r257506) that required depending kernel changes.  I'm also
avoiding very recent changes that have not had a long enough time to be
validated in current.

This is "make tinderbox" clean.

r256735	sh: Remove one syscall when waiting for a foreground job.
r257399	sh: Allow trapping SIGINT/SIGQUIT after ignore because of '&'.
r257504	sh: Reorder union node to reduce its size on 64-bit platforms.
r257920	sh: Add a test case for would-be assignments that are not due to quoting.
r257929	sh: Properly quote alias output from command -v.
r258489	sh: Add tests for the </dev/null implicit in a background command.
r258533	sh: Add more tests for the </dev/null implicit in a background command.
r258535	sh: Make <&0 disable the </dev/null implicit in a background command.
r258776	sh: Prefer memcpy() to strcpy() in most cases. Remove the scopy macro.
r259047	sh: Split set -x output into a separate function.
r259210	Migrate tools/regression/bin/ tests to the new layout.
r259844	sh: Remove an unused variable.
r259846	sh: Initialize OPTIND=1 even if it came from the environment.
r259874	sh: Simplify code related to PPID variable.
r259946	sh: Don't check input for non-whitespace if history is disabled.
r260246	sh(1): Discourage use of -e.
r260506	Run the sh(1) and test(1) tests as unprivileged.
r260586	Mark the bin/pax tests as requiring perl.
r260634	Use TAP_TESTS_PERL to register the legacy_test in bin/pax.
r260635	Replace hand-crafted Kyuafiles with automatic generation.
r260654	sh: Remove SIGWINCH handler and just check for resize before every read.
r261121	sh: Add test for nested alias.
r261125	sh: Solve the alias recursion problem in a less hackish way.
r261141	sh: Do not depend on parse/execute split in new alias test.
r261160	sh: Add tests for alias names after another alias.
r261192	sh: Allow aliases to force alias substitution on the following word.
r262533	sh: Make expari() static.
r262565	sh: Do not corrupt internal representation if LINENO inner expansion fails.
r262697	sh: Simplify expari().

Reviewed by:	jilles
2014-03-09 17:04:31 +00:00
daichi
b42eb5b769 MFC r262467: sh: Add -h option to SYNOPSIS
Reviewed by:	jilles
2014-02-25 03:49:42 +00:00
wblock
0b6cdba729 MFC r260782:
-h and -H options backwards in manual page.
2014-01-20 23:23:29 +00:00
trasz
62bfe594df MFC r256838:
Don't test arrays for being NULL.

Sponsored by:	The FreeBSD Foundation
2014-01-02 13:59:23 +00:00
brueffer
add016b338 MFC: r257288
Clean up a spurious "." in SEE ALSO.

Approved by:	re (glebius)
2013-11-11 14:04:26 +00:00
des
7c51c89f48 MFH (r256332): remove extraneous \n
Approved by:	re (gjb)
2013-10-11 20:14:21 +00:00
des
1fb35847da By popular demand, move freebsd-version(1) from /libexec to /bin.
Approved by:	re (gjb)
2013-10-09 20:47:20 +00:00
trasz
8dd526065c Remove useless check - ki_loginclass is an array; can't be NULL.
CID:		1006559
Approved by:	re (kib)
MFC after:	2 weeks
Sponsored by:	FreeBSD Foundation
2013-10-04 16:08:44 +00:00
pluknet
3f9b259642 Sweep man pages replacing ad -> ada.
Approved by:	re (blackend)
MFC after:	1 week
X-MFC note:	stable/9 only
2013-10-01 18:41:53 +00:00
jilles
ad7328341a sh: Make return return from the closest function or dot script.
Formerly, return always returned from a function if it was called from a
function, even if there was a closer dot script. This was for compatibility
with the Bourne shell which only allowed returning from functions.

Other modern shells and POSIX return from the function or the dot script,
whichever is closest.

Git 1.8.4's rebase --continue depends on the POSIX behaviour.

Reported by:	Christoph Mallon, avg
2013-09-04 22:10:16 +00:00
jilles
ae84eb37bd sh: Fix race condition with signals and wait or set -T.
The change in r238888 was incomplete. It was still possible for a trapped
signal to arrive before the shell went to sleep (sigsuspend()) because a
check was missing or because the signal arrived before in_waitcmd was set.

On SMP, this bug sometimes caused the builtins/wait4.0 test to take 1 second
to execute; it then might or might not fail. On UP, the test almost always
failed.
2013-09-02 21:57:46 +00:00
jilles
6d50a40d08 sh: Simplify list() in the parser.
The erflag argument was only used by old-style (``) command substitutions.
We can remove it and handle the special case in the command substitution
code.
2013-08-30 20:50:28 +00:00
jilles
fc9c61263d sh: Separate out nbinary allocation into a function. 2013-08-30 20:37:52 +00:00
jilles
67c1af2857 sh: Use makename() where possible. 2013-08-30 20:13:33 +00:00
jilles
e60f4ea26b sh: Add a function for the case where one token is required in the parse. 2013-08-30 13:25:15 +00:00
jilles
8b7d6031e6 sh: Recognize "--" as end of options in type builtin.
This implementation makes minimal changes: command names starting with "-"
(other than "--") can still be queried normally.
2013-08-30 12:09:59 +00:00
jilles
415d59b5d8 sh: Cast -1 to pointer rather than pointer to variable of wrong type.
NEOF needs to be a non-null pointer distinct from valid union node pointers.
It is not dereferenced.

The new NEOF is much like SIG_ERR except that it is an object pointer
instead of a function pointer.

The variable tokpushback can now be static.
2013-08-30 10:45:02 +00:00
will
a52b9ca1d3 Add the ability to display the default FIB number for a process to the
ps(1) utility, e.g. "ps -O fib".

bin/ps/keyword.c:
	Add the "fib" keyword and default its column name to "FIB".

bin/ps/ps.1:
	Add "fib" as a supported keyword.

sys/compat/freebsd32/freebsd32.h:
sys/kern/kern_proc.c:
sys/sys/user.h:
	Add the default fib number for a process (p->p_fibnum)
	to the user land accessible process data of struct kinfo_proc.

Submitted by:	Oliver Fromme <olli@fromme.com>, gibbs
2013-08-26 23:48:21 +00:00
jilles
59cd1d2d27 sh: Recognize "--" as end of options in alias builtin.
Aliases starting with "-" (which are non-POSIX) will need to be preceded by
an alias not starting with "-" or the newly added "--".
2013-08-25 11:42:53 +00:00
jilles
e694117f82 sh: Disallow empty simple commands.
As per POSIX, a simple command must have at least one redirection,
assignment word or command word.

These occured in rare cases such as  eval "f()" .

The extension of allowing no commands inside { }, if, while, for, etc.
remains.
2013-08-25 10:57:48 +00:00
jilles
f53205fcfa sh: Reject ++ and -- in arithmetic.
POSIX does not require ++ and -- in arithmetic. It is probably more useful
to reject them than to treat ++x and --x as x silently.

Note that the behaviour of increment and decrement can be obtained via
(x+=1), ((x+=1)-1), (x-=1) and ((x-=1)+1).

PR:		bin/176444
2013-08-24 20:06:00 +00:00
jilles
739c3b84b8 sh: Do not prematurely discard stopped jobs in a wait builtin.
If a job is specified to 'wait', wait for it to complete. Formerly, in
interactive mode, the job was deleted if it stopped.

If no jobs are specified in interactive mode, 'wait' still waits for all jobs
to complete or stop.

In non-interactive mode, WUNTRACED is not passed to wait3() so stopped jobs
are not detected.

PR:		bin/181435
2013-08-24 09:57:32 +00:00
ken
c7af094e18 Expand the use of stat(2) flags to allow storing some Windows/DOS
and CIFS file attributes as BSD stat(2) flags.

This work is intended to be compatible with ZFS, the Solaris CIFS
server's interaction with ZFS, somewhat compatible with MacOS X,
and of course compatible with Windows.

The Windows attributes that are implemented were chosen based on
the attributes that ZFS already supports.

The summary of the flags is as follows:

UF_SYSTEM:	Command line name: "system" or "usystem"
		ZFS name: XAT_SYSTEM, ZFS_SYSTEM
		Windows: FILE_ATTRIBUTE_SYSTEM

		This flag means that the file is used by the
		operating system.  FreeBSD does not enforce any
		special handling when this flag is set.

UF_SPARSE:	Command line name: "sparse" or "usparse"
		ZFS name: XAT_SPARSE, ZFS_SPARSE
		Windows: FILE_ATTRIBUTE_SPARSE_FILE

		This flag means that the file is sparse.  Although
		ZFS may modify this in some situations, there is
		not generally any special handling for this flag.

UF_OFFLINE:	Command line name: "offline" or "uoffline"
		ZFS name: XAT_OFFLINE, ZFS_OFFLINE
		Windows: FILE_ATTRIBUTE_OFFLINE

		This flag means that the file has been moved to
		offline storage.  FreeBSD does not have any special
		handling for this flag.

UF_REPARSE:	Command line name: "reparse" or "ureparse"
		ZFS name: XAT_REPARSE, ZFS_REPARSE
		Windows: FILE_ATTRIBUTE_REPARSE_POINT

		This flag means that the file is a Windows reparse
		point.  ZFS has special handling code for reparse
		points, but we don't currently have the other
		supporting infrastructure for them.

UF_HIDDEN:	Command line name: "hidden" or "uhidden"
		ZFS name: XAT_HIDDEN, ZFS_HIDDEN
		Windows: FILE_ATTRIBUTE_HIDDEN

		This flag means that the file may be excluded from
		a directory listing if the application honors it.
		FreeBSD has no special handling for this flag.

		The name and bit definition for UF_HIDDEN are
		identical to the definition in MacOS X.

UF_READONLY:	Command line name: "urdonly", "rdonly", "readonly"
		ZFS name: XAT_READONLY, ZFS_READONLY
		Windows: FILE_ATTRIBUTE_READONLY

		This flag means that the file may not written or
		appended, but its attributes may be changed.

		ZFS currently enforces this flag, but Illumos
		developers have discussed disabling enforcement.

		The behavior of this flag is different than MacOS X.
		MacOS X uses UF_IMMUTABLE to represent the DOS
		readonly permission, but that flag has a stronger
		meaning than the semantics of DOS readonly permissions.

UF_ARCHIVE:	Command line name: "uarch", "uarchive"
		ZFS_NAME: XAT_ARCHIVE, ZFS_ARCHIVE
		Windows name: FILE_ATTRIBUTE_ARCHIVE

		The UF_ARCHIVED flag means that the file has changed and
		needs to be archived.  The meaning is same as
		the Windows FILE_ATTRIBUTE_ARCHIVE attribute, and
		the ZFS XAT_ARCHIVE and ZFS_ARCHIVE attribute.

		msdosfs and ZFS have special handling for this flag.
		i.e. they will set it when the file changes.

sys/param.h:		Bump __FreeBSD_version to 1000047 for the
			addition of new stat(2) flags.

chflags.1:		Document the new command line flag names
			(e.g. "system", "hidden") available to the
			user.

ls.1:			Reference chflags(1) for a list of file flags
			and their meanings.

strtofflags.c:		Implement the mapping between the new
			command line flag names and new stat(2)
			flags.

chflags.2:		Document all of the new stat(2) flags, and
			explain the intended behavior in a little
			more detail.  Explain how they map to
			Windows file attributes.

			Different filesystems behave differently
			with respect to flags, so warn the
			application developer to take care when
			using them.

zfs_vnops.c:		Add support for getting and setting the
			UF_ARCHIVE, UF_READONLY, UF_SYSTEM, UF_HIDDEN,
			UF_REPARSE, UF_OFFLINE, and UF_SPARSE flags.

			All of these flags are implemented using
			attributes that ZFS already supports, so
			the on-disk format has not changed.

			ZFS currently doesn't allow setting the
			UF_REPARSE flag, and we don't really have
			the other infrastructure to support reparse
			points.

msdosfs_denode.c,
msdosfs_vnops.c:	Add support for getting and setting
			UF_HIDDEN, UF_SYSTEM and UF_READONLY
			in MSDOSFS.

			It supported SF_ARCHIVED, but this has been
			changed to be UF_ARCHIVE, which has the same
			semantics as the DOS archive attribute instead
			of inverse semantics like SF_ARCHIVED.

			After discussion with Bruce Evans, change
			several things in the msdosfs behavior:

			Use UF_READONLY to indicate whether a file
			is writeable instead of file permissions, but
			don't actually enforce it.

			Refuse to change attributes on the root
			directory, because it is special in FAT
			filesystems, but allow most other attribute
			changes on directories.

			Don't set the archive attribute on a directory
			when its modification time is updated.
			Windows and DOS don't set the archive attribute
			in that scenario, so we are now bug-for-bug
			compatible.

smbfs_node.c,
smbfs_vnops.c:		Add support for UF_HIDDEN, UF_SYSTEM,
			UF_READONLY and UF_ARCHIVE in SMBFS.

			This is similar to changes that Apple has
			made in their version of SMBFS (as of
			smb-583.8, posted on opensource.apple.com),
			but not quite the same.

			We map SMB_FA_READONLY to UF_READONLY,
			because UF_READONLY is intended to match
			the semantics of the DOS readonly flag.
			The MacOS X code maps both UF_IMMUTABLE
			and SF_IMMUTABLE to SMB_FA_READONLY, but
			the immutable flags have stronger meaning
			than the DOS readonly bit.

stat.h:			Add definitions for UF_SYSTEM, UF_SPARSE,
			UF_OFFLINE, UF_REPARSE, UF_ARCHIVE, UF_READONLY
			and UF_HIDDEN.

			The definition of UF_HIDDEN is the same as
			the MacOS X definition.

			Add commented-out definitions of
			UF_COMPRESSED and UF_TRACKED.  They are
			defined in MacOS X (as of 10.8.2), but we
			do not implement them (yet).

ufs_vnops.c:		Add support for getting and setting
			UF_ARCHIVE, UF_HIDDEN, UF_OFFLINE, UF_READONLY,
			UF_REPARSE, UF_SPARSE, and UF_SYSTEM in UFS.
			Alphabetize the flags that are supported.

			These new flags are only stored, UFS does
			not take any action if the flag is set.

Sponsored by:	Spectra Logic
Reviewed by:	bde (earlier version)
2013-08-21 23:04:48 +00:00
jilles
e5ea815310 sh: Remove unnecessary reset functions.
These are already handled by exception handlers.
2013-08-16 20:24:41 +00:00
jilles
c433419763 sh: Recognize "--" as end of options in bg/fg/jobid builtins. 2013-08-16 13:56:43 +00:00
jilles
c71191e0d9 sh: Recognize "--" as end of options in local builtin. 2013-08-14 21:59:48 +00:00
jilles
79360f25fc sh: Allow a lone redirection before '|', ';;' or ';&'.
Example: </dev/null | :

PR:		181240
MFC after:	1 week
2013-08-14 19:34:13 +00:00
trasz
9d86b16a9e Add -c flag to pgrep(1) and pkill(1), to match login classes.
MFC after:	1 month
2013-08-09 08:38:51 +00:00
jilles
44afceea6d sh: Remove an incorrect comment. 2013-07-25 20:50:35 +00:00
jilles
45d56acf59 sh: Remove #define MKINIT.
MKINIT only served for the removed mkinit. Many variables can be static now.
2013-07-25 19:48:15 +00:00
jilles
0ad2a46f33 sh: Remove mkinit.
Replace the RESET blocks with regular functions and a reset() function that
calls them all.

This code generation tool is unusual and does not appear to provide much
benefit. I do not think isolating the knowledge about which modules need to
be reset is worth an almost 500-line build tool and wider scope for
variables used by the reset functions.

Also, relying on reset functions is often wrong: the cleanup should be done
in exception handlers so that no stale state remains after 'command eval'
and the like.
2013-07-25 15:08:41 +00:00