Instead of changing the whole course to another POSIX-permitted way
for consistency and uniformity I decide to completely ignore missing
regex fucntionality and focus on fixing bugs in what we have now,
too many small obstacles we have choicing other way, counting ports.
Corresponding libc changes are backed out in r302824.
On machines where SIZE_T_MAX exceeds OFF_MAX (signed 64-bit), permit seeking
character devices to negative off_t values. This enables dd(1) to interact
with kernel KVA in /dev/kmem on amd64, for example.
Sponsored by: EMC / Isilon Storage Division
This is the same issue as r297997, but was missed in it.
The WARNS value changes between 'build-tools' (MK_WARNS=no) and
'everything' resulting in a rebuild of this file.
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division
The fix in r300649 was not sufficient to convince Coverity that the
buffer was NUL terminated, even with the buffer pre-zeroed. Swap
the size and nmemb arguments to fread() so that a valid lenght is
returned, which we can use to terminate the string in the buffer
at the correct location. This should also quiet the complaint about
the return value of fread() not being checked.
Reported by: Coverity
CID: 1019054, 1009614
MFC after: 1 week
The first file in these lists will generate everything else so only
it should be getting a .meta file. With bmake's missing=yes meta
feature these would otherwise cause a rebuild without the
.NOMETA hint.
Sponsored by: EMC / Isilon Storage Division
ki_flag and ki_tdflag have been 'long', not 'int', since 2000 and 2005,
respectively.
Submitted by: Shawn Wills <swills at isilon dot com>
Sponsored by: EMC / Isilon Storage Division
case of \c in the prompt format string is a no-op. We already passed
this test at the top of the loop, and i has not yet been incremented in
this path. Change this test to (i < PROMPTLEN - 2).
Reported by: Coverity
CID: 1008328
Reviewed by: cem
MFC after: 1 week
This will make things more flexible if the program path changes in the future,
and the test in and of itself doesn't call /usr/bin/bc -- it just calls bc
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
There is no need to to call strdup() on the value returned by fmt().
The latter calls fmt_argv() which always returns a dynamically
allocated string, and calling strdup() on that leaks the memory
allocated by fmt_argv(). Wave some const magic on ki_args and
ki_env to make the direct assignment happy. This requires a tweak
to the asprintf() case to avoid a const vs. non-const mismatch.
Reported by: Coverity
CID: 1011370
MFC after: 1 week
Replace the magic constant 127 in the loop interation count with
"PROMPTLEN - 1".
gethostname() is not guaranteed to NUL terminate the destination
string if it is too short. Decrease the length passed to gethostname()
by one, and add a NUL at the end of the buffer to make sure the
following loop to find the end of the name properly terminates.
The default: case is the likely cause of Coverity CID 1008328. If
i is 126 at the top of the loop interation where the default case
is triggered, i will be incremented to 127 by the default case,
then incremented to 128 at the top of the loop before being compared
to 127 (PROMPTLENT - 1) and terminating the loop. Then the NUL
termination code after the loop will write to ps[128]. Fix by
checking for overflow before incrementing the index and storing the
second character in the buffer.
These fixes are not guaranteed to satisfy Coverity. The code that
increments i in the 'h'/'H' and 'w'/'W' cases may be beyond its
capability to analyze, but the code appears to be safe.
Reported by: Coverity
CID: 1008328
Reviewed by: jilles, cem
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D6482
- Use arc4random_uniform(3).
- Prevent a segmentation fault when ed receives a signal
while being in getpass(). [1]
Obtained from: OpenBSD [1] (CVS Rev. 1.15)
MFC after: 2 weeks
When getline(3) in 2009 was added a _WITH_GETLINE guard has also been added.
This rename is made in preparation for the removal of this guard
Obtained from: NetBSD
after r298107
Summary of changes:
- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)
Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.
MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
If the hostname is empty and \h is used in $PS1,
the remainder of the prompt following \h will be empty.
Likewise for $PWD and \w. Fix it.
Reviewed by: jilles
MFC after: 1 week
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D6188
Fix a related typo while here.
Note, this change results in the Kyuafile inclusion in the runtime
package, which needs to be fixed, however addresses the PR as far
as I can tell in my tests.
PR: 209114
Submitted by: ngie
Sponsored by: The FreeBSD Foundation
is invalid. It is used to find the size of allocated memory. As malloc may
allocate memory with mmap it will fail to take this memory into account.
Obtained from: brooks
This avoids 'build command changed' due to CFLAGS/CC changes during the
normal build. Without this the build-tools targets end up rebuilding
for the *target* rather than keeping the native versions built in
build-tools.
Sponsored by: EMC / Isilon Storage Division
At least for ncurses this fixes a build error due to it trying to run
'ccache --version' to work around a gcc 5 bug using the fix in r287205.
Sponsored by: EMC / Isilon Storage Division
libraries (such as libthr) which maintain their own signal state. This
change adds the tcsh SAVESIGVEC option to save and restore the sigvecs for
the signals the child modifies before it execs.
Reviewed by: kib, rwatson
Reported by: kib
breaking the ABI. Special value is stored in the lock pointer to
indicate shared lock, and offline page in the shared memory is
allocated to store the actual lock.
Reviewed by: vangyzen (previous version)
Discussed with: deischen, emaste, jhb, rwatson,
Martin Simmons <martin@lispworks.com>
Tested by: pho
Sponsored by: The FreeBSD Foundation
(when they actually get committed, that is), and might also come in handy
in other situations.
Reviewed by: wblock@ (man page)
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.
Sponsored by: EMC / Isilon Storage Division
The intent of the previous code in that case was to force
an explicit write, but the implementation was incorrect, and
as a result the write was never performed. This new implementation
instead uses ftruncate(2) to extend the file with a trailing hole.
Also introduce regression tests for these cases.
PR: 189284
(original PR whose fix introduced this bug)
PR: 207092
Differential Revision: D5248
Reviewed by: sobomax,kib
MFC after: 2 weeks
Note: tcsh(1) has a MK_TCSH=no test, so this should be a separate
package, which requires pre-install/post-install scripts, to be
added later.
Sponsored by: The FreeBSD Foundation
The symbols of libiconv from ports were changed to
have prefixed.
Since we have iconv in our libc these days, we don't
need it on 10.X and later. However, 9.X still need
this.
Spotted by: Yoshihiko Sarumaru
MFC after: 1 days
Builtins (including variable assignments without command word), function
calls and redirected compound commands need to restore file descriptors
to their original state after execution. This is handled by allocating a
redirtab structure. These mallocs and frees show up heavily in pmcstat.
Only allocate a redirtab if there are actually redirections and maintain a
count of how many levels of REDIR_PUSH there are without redirtabs.
A simple loop without external programs like
sh -c 'i=0; w=$(printf %0100d 7); while [ "$i" -lt 1000000 ]; do
i=$((i+1)); done'
is over 25% faster on an amd64 bhyve VM.
The linear search using strcmp() shows up in pmcstat for several percent.
Split the operators into lengths and whether they start with '-' and compare
bytes using == instead of strcmp().
A simple test
sh -c 'i=0; w=$(printf %0100d 7); while [ "$i" -lt 1000000 ]; do
v=$(printf %sx%s "$w" "$w"); i=$((i+1)); done'
is over 4% faster on an amd64 bhyve VM.
Forbid (temporary or permanent) modifications of the strings in NARG nodes
during expansion.
Tilde expansion now needs to copy the username for the terminating '\0'.
Per POSIX, 'cd -' should use the OLDPWD shell variable, not internal state.
This variable is normally exported.
Also, if OLDPWD is not set, fail 'cd -' instead of changing to the current
directory.
Some variables like PATH call a function when modified. Make sure to call
this also when leaving a function where such a variable was made local.
Make sure to restore local variables before shellparam, so getopts state is
not clobbered.
Redirect 'cd -' output to /dev/null since POSIX requires it to write the new
directory name even if not interactive, but we currently only write it if
interactive.
Command substitutions containing a single simple command and here-document
expansion are performed in a subshell environment, but may not fork. Any
modified state of the shell environment should be restored afterward.
The state that OPTIND=1 had been done was not saved and restored here.
Note that the other parts of shellparam need not be saved and restored,
since they are not modified in these situations (a fork is done before such
modifications).
With the new expand.c code, the intermediate representation passed to the
pathname generation code only contains CTLESC, not CTLQUOTEMARK.
CTLQUOTEMARK now only occurs in the text of NARG nodes (output of the
parser).
This avoids the need to add and remove CTLESC bytes if pathname generation
will not be performed (set -f).
Side effect: the order of operations is slightly different: pathname
generation in ${$+* $(CMD)} will not see filesystem changes from CMD.
This simplifies the code and should be faster in some cases.
Side effect: the order of operations is different so that the value of IFS
used when IFS is modified during expansion (${IFS:=...}, ${IFS=...} or
$((...IFS=...))) may be different. Note that this order is highly unportable
between shells.
Latest update of locales introduced abbreviated month that follows the regionale
rules meaning that they can be of variable length instead of being arbitrary
truncated to top 3 characters.
To fix alignement, ls now computes the visible length of the abbreviated month,
pads the shorter month with spaces in order to make sure everything is properly
aligned
Reviewed by: ache, ed, jilles
Differential Revision: https://reviews.freebsd.org/D4239
kyua 0.11's version of report-junit was rendering non-printable characters
Upgrade to kyua 0.12 to obtain a fixed version of the command
Output verified with python 2.7.10's xml.dom.minidom module
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division
Unfortunately filemon/meta mode tracks all indirect dependencies here
since ld(1) is reading libelf when linking in libkvm. Churn would be
reduced if this was able to be limited to direct dependencies.
Sponsored by: EMC / Isilon Storage Division
Tracking these leads to situations where meta mode will consider the
file to be out of date if /bin/sh or /bin/ln are newer than the source
file. There's no reason for meta mode to do this as make is already
handling the rebuild dependency fine.
Sponsored by: EMC / Isilon Storage Division
It included libutil.h for setproctitle(3), which was moved from libutil to libc
in r65353 in 2000.
Reviewed by: gshapiro [sendmail change]
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D4261
edition 2013. No need anymore to disable the protection if one set
the POXILY_CORRECT environment variable.
Reviewed by: imp
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D4092
bin/dd/tests
Ensure fdescfs is mounted on /dev/fd/ for the length testcase as it's used
in validating the characters read from /dev/zero
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
netbsd-tests.test.mk (r289151)
- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
This simplifies the code (e.g. allowing use of qsort(3) instead of a
hand-rolled mergesort) and should have better cache properties.
The waste of unused args arrays after resizes is approximately the same as
the savings from getting rid of the next pointers.
At the same time, remove a piece of global state and move some duplicated
code into a function.
and move from the pattern of:
.if ${MK_FOO} != "no"
SUBDIR+= bar
.endif
to
SUBDIR.${MK_FOO}+= bar
since we know that MK_FOO is always either yes or no and the latter
form is easier to follow and much shorter. Various exception to this
pattern dealt with on an ah-hoc basis.
Discussed on arch@ a while ago.
^/user/ngie/more-tests.
- Additional testcases added:
-- ls -D
-- ls -F
-- ls -H
-- ls -L
-- ls -R
-- ls -S
-- ls -T
-- ls -b
-- ls -d
-- ls -f
-- ls -g
-- ls -h
-- ls -i
-- ls -k
-- ls -l
-- ls -m
-- ls -n
-- ls -o
-- ls -p
-- ls -q/ls -w
-- ls -r
-- ls -s
-- ls -t
-- ls -u
-- ls -y
- Socket file creation is limited to the ls -F testcase, greatly speeding up
the test process
- The ls -C testcase was made more robust by limiting the number of columns
via COLUMNS and by dynamically formulating the columns/lines.
- Add `atf_test_case` before all testcase `head` functions.
X-MFC with: r284388, r288330, r288423
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Shell syntax is too complicated to detect command substitution and unquoted
operators reliably without implementing much of sh's parser. Therefore, have
sh do this detection.
While changing sh's support anyway, also read input from a pipe instead of
arguments to avoid {ARG_MAX} limits and improve privacy, and output count
and length using 16 instead of 8 digits.
The basic concept is:
execl("/bin/sh", "sh", "-c", "freebsd_wordexp ${1:+\"$1\"} -f "$2",
"", flags & WRDE_NOCMD ? "-p" : "", <pipe with words>);
The WRDE_BADCHAR error is still implemented in libc. POSIX requires us to
fail strings containing unquoted braces with code WRDE_BADCHAR. Since this
is normally not a syntax error in sh, there is still a need for checking
code in libc, we_check().
The new we_check() is an optimistic check that all the characters
<newline> | & ; < > ( ) { }
are quoted. To avoid duplicating too much sh logic, such characters are
permitted when quoting characters are seen, even if the quoting characters
may themselves be quoted. This code reports all WRDE_BADCHAR errors; bad
characters that get past it and are a syntax error in sh return WRDE_SYNTAX.
Although many implementations of WRDE_NOCMD erroneously allow some command
substitutions (and ours even documented this), there appears to be code that
relies on its security (codesearch.debian.net shows quite a few uses).
Passing untrusted data to wordexp() still exposes a denial of service
possibility and a fairly large attack surface.
Reviewed by: wblock (man page only)
MFC after: 2 weeks
Relnotes: yes
Security: fixes command execution with wordexp(untrusted, WRDE_NOCMD)
POSIX requires this to prevent entering function definitions in history but
this implementation does nothing except retain the option's value. In ksh88,
function definitions were usually entered in the history file, even when
they came from ~/.profile and the $ENV file, to allow displaying their
definitions.
This is also the first option that does not have a letter.
The initial check for a matching ] was incorrect if a ] may be consumed by a
[:class:]. The subsequent loop assumed that there must be a ].
Remove the initial check and make the loop cope with a missing ].
Found with afl-fuzz.
MFC after: 1 week
An invalid substitution like ${var@} does not cause a parse error but is
stored in the intermediate representation, to be written as part of the
error message. If there is a CTL* byte in the stored part, this confuses
some code such as the code to skip an unused alternative such as in
${var-alternative}.
To keep things simple, do not store CTL* bytes.
Found with afl-fuzz.
MFC after: 1 week
The negative value was not expected and generated the low 8 bits as a byte,
which may be an invalid character encoding.
The final shift in creating the negative value was undefined as well.
Make the temporary variable unsigned to fix this.
Correctly escape literal % for display
This fixes segfaults in 32bit arches caused by r285734
Reviewed by: ngie
Approved by: dim
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3191
* Add whitespace trimming to some fields (username, group, size, inode, blocks) to avoid whitespace in JSON strings
* fix -m mode, was invalid JSON (repeated keys), and was missing outer array container
* in -n mode, numeric uids and gids were returned as strings
Approved by: eadler (mentor)
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D2854
The message text is from cp, which has had a nicer message for this since
2007 (PR bin/50656).
As with cp, the exit status changes from 64 to 1.
PR: 201083
MFC after: 1 week
Fix shifts of possibly negative numbers found with ubsan and avoid signed
integer overflow when hashing an extremely long command name.
MFC after: 1 week
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
using the color of symbolic link, the color is determined by the link target.
This behavior was quite confusing.
Looking at the file history, it looks like that r203665 intends to fix this
but the issue was never actually fixed.
Fix this by not setting FTS_COMFOLLOW when color is requested like what was
done in r203665.
MFC after: 2 weeks
This is also implemented in at least GNU coreutils cp.
While here also improve the '-l' handling to not open(2) the source file as
it does not actually need the descriptor.
Sponsored by: EMC / Isilon Storage Division
In this order, it won't try to recurse into a directory for which
it doesn't have permission, before changing that permission.
This follows an existing behavior in other BSDs, linux, OS X.
Obtained from: NetBSD
interpreted as a number, which checks the file's modification time and
use that as the date/time value.
This improves compatibility with GNU coreutils's version of time(1).
MFC after: 2 weeks
In an attempt to improve performance, cp reordered directories first
(although the comment says directories last). This is not effective with new
UFS layout policies.
The sorting reorders multiple arguments passed to cp, which may be
undesirable.
Additionally, the comparison function does not induce a total order. Per
POSIX, this causes undefined behaviour in qsort().
NetBSD removed the sorting in 2009.
On filesystems that return directory entries in hash/btree order, sorting by
d_fileno before statting improves performance on large directories. However,
this can only be implemented in fts(3).
PR: 53475
Reviewed by: bde (in 2004)
MFC after: 1 week
interactive.
I added the interactive check in r208881 to be safe, but in actual use
(scripts in set -m mode) passing along SIGINT seems best.
Discussed with: bdrewery
/usr/src/bin/ed/glbl.c:64:36: error: logical not is only applied to
theleft hand side of comparison [-Werror=logical-not-parentheses]
Obtained from: Dragonfly (1fff89cbaeaa43af720a1f23d9c466b756dd8a58)
MFC After: 1 month
The parser considered 'trap exit INT' to reset the default for both EXIT and
INT. This beahvior is not POSIX compliant. This was avoided if a value was
specified for 'exit', but then disallows exiting with the signal received. A
possible workaround is using ' exit'.
However POSIX does allow this type of behavior if the parameters are all
integers. Fix the handling for this and clarify its support in the manpage
since it is specifically allowed by POSIX.
Differential Revision: https://reviews.freebsd.org/D2325
Reviewed by: jilles
MFC after: 2 weeks
-O Force the archive to be one volume. If a volume ends prematurely, pax will
not prompt for a new volume.
PR: 198481
Submitted by: Sevan Janiyan
Reviewed by: allanjude (doc)
The previous 12h AM/PM format was perplexing as it didn't follow the
locale of the user and was a minor annoyance to FreeBSD users coming
from Linux. Additionally, the man page was incorrect about the strftime
format.
There are three time formats that may be displayed in the STARTED
column depending on the age of the process. Below is an example.
For a process started at 14:30 on Monday 16 March 2015, the following
formats may be used:
14:30 for process < 24h old (24h Timestamp)
Mon14 for process > 24h, < 1 week old (Weekday Hour)
16Mar15 for process > 1 week old (Day Month Year)
Differential Revision: https://reviews.freebsd.org/D1620
Reviewed by: brd
Approved by: trasz
r278742:
Simplify jail_name_to_jid and try to be more fault tolerant when scanning for
the jail ID (poll up to 10 times for the jail IDs to become available)
If the scan fails, the code will fall through and fail as it does with Jenkins
today
r278636:
Parameterize out the amount of sleep done in each test
Set the value in each test to a different amount to avoid potential
side-effects with other instances of the test (or lingering processes) still
being present on the system
r278633:
Refactor the tests
1. `id -u` -> 0 is now only checked once; the entire test script is now skipped
if this assertion is violated
2. De-dent whitespace, based on 1.
3. Only setup the symlink for $sleep once at the top of the script, and tear it
down once at the bottom of the script
the jail ID (poll up to 10 times for the jail IDs to become available)
If the scan fails, the code will fall through and fail as it does with Jenkins
today
Set the value in each test to a different amount to avoid potential
side-effects with other instances of the test (or lingering processes) still
being present on the system
1. `id -u` -> 0 is now only checked once; the entire test script is now skipped
if this assertion is violated
2. De-dent whitespace, based on 1.
3. Only setup the symlink for $sleep once at the top of the script, and tear it
down once at the bottom of the script
- Add special test to detect the case of -1 * INTMAX_MIN
- Protect against elimination of the test division by the optimizer
Garrett Cooper noticed that the overflow checks were incomplete, and Bruce
Evans suggested the use of the "volatile" qualifier to counter the effect
of the undefined behaviour, when the prior multiplication caused overflow,
and he also suggested improvements to the comments.
Reviewed by: bde
MFC after: 1 week
The core kernel part is patch file utimes.2008.4.diff from
pluknet@FreeBSD.org. I updated the code for API changes, added the manual
page and added compatibility code for old kernels. There is also audit and
Capsicum support.
A new UTIME_* constant might allow setting birthtimes in future.
Differential Revision: https://reviews.freebsd.org/D1426
Submitted by: pluknet (partially)
Reviewed by: delphij, pluknet, rwatson
Relnotes: yes
Some users build FreeBSD as non-root in Perforce workspaces. By default,
Perforce sets files read-only unless they're explicitly being edited.
As a result, the -f argument must be used to cp in order to override the
read-only flag when copying source files to object directories. Bare use of
'cp' should be avoided in the future.
Update all current users of 'cp' in the src tree.
Reviewed by: emaste
MFC after: 1 week
Sponsored by: Spectra Logic
EXP_REDIR was supposed to generate pathnames in redirection if exactly one
file matches, as permitted but not required by POSIX in interactive mode. It
is unlikely this will be implemented.
No functional change is intended.
MFC after: 1 week
Commands like 'export -p', 'set' and 'trap', and tracing enabled via 'set
-x' generate output suitable as shell input by adding quotes as necessary.
If there are control characters other than newline or invalid UTF-8
sequences, use $'...' and \OOO to display them safely.
The resulting output is not parsable by a strict POSIX.1-2008 shell but sh
from FreeBSD 9.0 and newer and many other shells can parse it.
feature is to quisce the system before suspend.
Stop is implemented by reusing the thread_single(9) with the special
mode SINGLE_ALLPROC. SINGLE_ALLPROC differs from the existing
single-threading modes by allowing (requiring) caller to operate on
other process. Interruptible sleeps for !TDF_SBDRY threads are
suspended like SIGSTOP does it, instead of aborting the sleep, like
SINGLE_NO_EXIT, to avoid spurious EINTRs on resume.
Provide debugging sysctl debug.stop_all_proc, which causes total stop
and suspends syncer, while waiting for variable reset for resume. It
is used for debugging; should be removed after the real use of the
interface is added.
In collaboration with: pho
Discussed with: avg
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
