r276669:
Integrate bin/cat/tests from NetBSD into atf/kyua
Sponsored by: EMC / Isilon Storage Division
r276723:
Install d_align.{in,out} for the :align test
Pointyhat to: me
Enable bin/ls testcases disabled previously because of issues with how
kyua 0.11's version of report-junit was rendering non-printable characters
Upgrade to kyua 0.12 to obtain a fixed version of the command
Output verified with python 2.7.10's xml.dom.minidom module
Sponsored by: EMC / Isilon Storage Division
r284105:
Cleanup some indentation issues.
r284106:
Implement '-s' to copy as symlink, similar to the current -l link(2)
handling.
r284163:
Cleanup some style(9) issues.
Relnotes: yes
r269902:
Convert bin/sh/tests to ATF
The new code uses a "test discovery mechanism" to determine
what tests are available for execution
The test shell can be specified via:
kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh
Sponsored by: EMC / Isilon Storage Division
Approved by: jmmv (mentor)
Reviewed by: jilles (maintainer)
r270101 (by jilles):
sh: Don't hardcode relative paths in the tests stderr files.
These paths have had to be adjusted to changes in the testsuite runner
several times, so modify the tests to remove the need for such adjustment.
A cp in functional_test.sh is now unneeded, but this matters little in
performance.
Integrate the tests from lib/libarchive, usr.bin/cpio, and usr.bin/tar in to
the FreeBSD test suite
functional_test.sh was ported from bin/sh/tests/functional_test.sh, as a
small wrapper around libarchive_test, bsdcpio_test, and bsdtar_test provided
by upstream.
A handful of testcases in lib/libarchive/tests have been disabled as they
were failing when run with kyua test (see BROKEN_TESTS in
lib/libarchive/tests/Makefile)
As a sidenote: this removes the check/test targets from the Makefiles as they
don't match the pattern used in the rest of the FreeBSD test suite.
Sponsored by: EMC / Isilon Storage Division
Conflicts:
lib/libarchive/test
usr.bin/cpio/test
Protecting against rm -rf / is now POSIXLY_CORRECT per posix 1003.1
edition 2013. No need anymore to disable the protection if one set
the POXILY_CORRECT environment variable.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D4092
Integrate contrib/netbsd-tests/bin/dd into the FreeBSD test suite as
bin/dd/tests
Ensure fdescfs is mounted on /dev/fd/ for the length testcase as it's used
in validating the characters read from /dev/zero
Sponsored by: EMC / Isilon Storage Division
r288330:
Add initial testcases for bin/ls
Sponsored by: EMC / Isilon Storage Division
r288423:
Skip the B_flag testcase to stop blowing up freebsd-current@ with
"test failure emails" because kyua report-jenkins doesn't properly
escape non-printable chars
r288678:
Merge additional testcases and improvements to bin/ls/ls_tests from
^/user/ngie/more-tests.
- Additional testcases added:
-- ls -D
-- ls -F
-- ls -H
-- ls -L
-- ls -R
-- ls -S
-- ls -T
-- ls -b
-- ls -d
-- ls -f
-- ls -g
-- ls -h
-- ls -i
-- ls -k
-- ls -l
-- ls -m
-- ls -n
-- ls -o
-- ls -p
-- ls -q/ls -w
-- ls -r
-- ls -s
-- ls -t
-- ls -u
-- ls -y
- Socket file creation is limited to the ls -F testcase, greatly speeding up
the test process
- The ls -C testcase was made more robust by limiting the number of columns
via COLUMNS and by dynamically formulating the columns/lines.
- Add `atf_test_case` before all testcase `head` functions.
Sponsored by: EMC / Isilon Storage Division
r288905:
Add some more syncs to quiesce the filesystem after creating the
files to see if this fixes deterministic Jenkin failures
r288906:
Explicitly set BLOCKSIZE to 512 in the environment
r288907:
Call sync consistently using atf_check
Remove superfluous sync's
r289102:
Remove all of the syncs
They're unnecessary as shown by further testing on my VM
Requested by: jhb
Shell syntax is too complicated to detect command substitution and unquoted
operators reliably without implementing much of sh's parser. Therefore, have
sh do this detection.
While changing sh's support anyway, also read input from a pipe instead of
arguments to avoid {ARG_MAX} limits and improve privacy, and output count
and length using 16 instead of 8 digits.
The basic concept is:
execl("/bin/sh", "sh", "-c", "freebsd_wordexp ${1:+\"$1\"} -f "$2",
"", flags & WRDE_NOCMD ? "-p" : "", <pipe with words>);
The WRDE_BADCHAR error is still implemented in libc. POSIX requires us to
fail strings containing unquoted braces with code WRDE_BADCHAR. Since this
is normally not a syntax error in sh, there is still a need for checking
code in libc, we_check().
The new we_check() is an optimistic check that all the characters
<newline> | & ; < > ( ) { }
are quoted. To avoid duplicating too much sh logic, such characters are
permitted when quoting characters are seen, even if the quoting characters
may themselves be quoted. This code reports all WRDE_BADCHAR errors; bad
characters that get past it and are a syntax error in sh return WRDE_SYNTAX.
Although many implementations of WRDE_NOCMD erroneously allow some command
substitutions (and ours even documented this), there appears to be code that
relies on its security (codesearch.debian.net shows quite a few uses).
Passing untrusted data to wordexp() still exposes a denial of service
possibility and a fairly large attack surface.
This is also a MFC of r286830 to reduce conflicts. I changed the code
somewhat to avoid changes from r286941; in particular, WRDE_BADVAL can still
only be returned if WRDE_UNDEF was passed.
Relnotes: yes
Security: fixes command execution with wordexp(untrusted, WRDE_NOCMD)
Use exit() instead of return in main(). The difference in practice
is subtle: C standard requires the language runtime to make return
of int from main() behave like calling exit(), and in FreeBSD we do:
exit(main(argc, argv, env))
In lib/csu/${ARCH}/crt1.c, so the real difference is using exit()
explicitly would use an additional stack frame.
Note however, if there is a on stack pointer is the last reference
of an allocated memory block, returning from the function would,
technically, result in a memory leak because we lost the last
reference to the memory block, and calling exit() from C runtime
could potentionally overwrite that stack frame that used to belong
to the main() function.
In practice, this is normally Okay because eventually the kernel
would tear down the whole address space that belongs to the process
in the _exit(2) system call, but the difference could confuse
compilers (which may want to do stack overflow checks) and static
analyzers.
Replacing return with exit() in main() allows compilers/static
analyzers to correctly omit or generate the right warnings when
they do not treat main() specifically. With the current version
of clang on FreeBSD/amd64, use of exit() would result in slightly
smaller code being generated and eliminated a false positive
warning of memory leak.
The initial check for a matching ] was incorrect if a ] may be consumed by a
[:class:]. The subsequent loop assumed that there must be a ].
Remove the initial check and make the loop cope with a missing ].
Found with afl-fuzz.
substitution error.
An invalid substitution like ${var@} does not cause a parse error but is
stored in the intermediate representation, to be written as part of the
error message. If there is a CTL* byte in the stored part, this confuses
some code such as the code to skip an unused alternative such as in
${var-alternative}.
To keep things simple, do not store CTL* bytes.
Found with afl-fuzz.
non-directory.
The message text is from cp, which has had a nicer message for this since
2007 (PR bin/50656).
As with cp, the exit status changes from 64 to 1.
PR: 201083
date(1): Make -r behave like GNU's version when the option can not be
interpreted as a number, which checks the file's modification time and
use that as the date/time value.
This improves compatibility with GNU coreutils's version of date(1).
In an attempt to improve performance, cp reordered directories first
(although the comment says directories last). This is not effective with new
UFS layout policies.
The sorting reorders multiple arguments passed to cp, which may be
undesirable.
Additionally, the comparison function does not induce a total order. Per
POSIX, this causes undefined behaviour in qsort().
NetBSD removed the sorting in 2009.
On filesystems that return directory entries in hash/btree order, sorting by
d_fileno before statting improves performance on large directories. However,
this can only be implemented in fts(3).
PR: 53475
Reviewed by: bde (in 2004)
Implement pax -O option to permit limiting a PAX archive to a single volume.
-O Force the archive to be one volume. If a volume ends prematurely, pax will
not prompt for a new volume.
PR: 198481
parameters.
Per Austin Group issue #459, shifting zero positional parameters may or may
not be considered an operand error (which causes the shell to exit in most
cases).
EXP_REDIR was supposed to generate pathnames in redirection if exactly one
file matches, as permitted but not required by POSIX in interactive mode. It
is unlikely this will be implemented.
No functional change is intended.
redirection.
EXP_REDIR was not being checked for while expanding positional parameters in
redirection, so CTL* bytes were not being prefixed where they should be.
64-bit systems.
Currently, there can be no more than INT_MAX positional parameters. Make
sure to treat all higher ones as unset to avoid incorrect results and
crashes.
On 64-bit systems, our atoi() takes the low 32 bits of the strtol() and
sign-extends them.
On 32-bit systems, the call to atoi() returned INT_MAX for too high values
and there is not enough address space for so many positional parameters, so
there was no issue.
PR: 195918
rm -rf can fail sometimes with an error from fts_read. Make it
honor fflag to ignore fts_read errors, but stop deleting from
that directory because no further progress can be made.
When building a kernel with a high -j value on a high core count
machine, during the cleanobj phase we can wind up doing multiple
rm -rf at the same time for modules that have subdirectories. This
exposed this race (sometimes) as fts_read can return an error if
the directory is removed by another rm -rf. Since the intent of
the -f flag was to ignore errors, even if this was a bug in
fts_read, we should ignore the error like we've been instructed
to do.
Approved by: re (kib)
Sponsored by: The FreeBSD Foundation
