Note that this depends on r206145 for allowing pattern match characters to
have their special meaning inside a double-quoted expansion like "${v%pat}".
PR: bin/117748
Exp-run done by: erwin (with some other sh(1) changes)
* remove the backslash from \} inside double quotes inside +-=?
substitutions, e.g. "${$+\}a}"
* maintain separate double-quote state for ${v#...} and ${v%...};
single and double quotes are special inside, even in a double-quoted
string or here document
* keep track of correct order of substitutions and arithmetic
This is different from dash's approach, which does not track individual
double quotes in the parser, trying to fix this up during expansion.
This treats single quotes inside "${v#...}" incorrectly, however.
This is similar to NetBSD's approach (as submitted in PR bin/57554), but
recognizes the difference between +-=? and #% substitutions hinted at in
POSIX and is more refined for arithmetic expansion and here documents.
PR: bin/57554
Exp-run done by: erwin (with some other sh(1) changes)
Redirection errors on subshells already did not abort the shell because
the redirection is executed in the subshell.
Other shells seem to agree that these redirection errors should not abort
the shell.
Also ensure that the redirections will be cleaned up properly in cases like
command eval '{ shift x; } 2>/dev/null'
Example:
{ echo bad; } </var/empty/x; echo good
- A couple of tests to check if the layout of the generated calenders
is correct.
- A couple of tests to see if impossible combinations for -3, -A,
-m, -y etc properly abort.
- A couple of test to confirm that the order of -A, -B, -3 etc give
the right number of months.
Although simple commands without a command word (only assignments and/or
redirections) are much like special builtins, POSIX and most shells seem to
agree that redirection errors should not abort the shell in this case. Of
course, the assignments persist and assignment errors are fatal.
To get the old behaviour portably, use the ':' special builtin.
To get the new behaviour portably, given that there are no assignments, use
the 'true' regular builtin.
We currently ignore readonly status for assignments before regular builtins
and external programs (these assignments are not persistent anyway), so just
check that the readonly variable really is not changed.
The test depends on the command builtin changes for 'command :'.
These expansions, which were already in the Bourne shell, work correctly for
the most part. The testcases are only about the parts that already work
correctly.
They are mainly about expansions in here documents but because all the
testcases are in $() command substitution, we also test that $() command
substitution is recursively parsed (or very close to it).
* avoid unnecessary fork
* allow executing builtins via command
* executing a special builtin via command removes its special properties
Obtained from: NetBSD (parts)
snprintf(3) doesn't set errno in the tested cases.
- If the same argument reference (for example %1) was specified more than
once, the command didn't necessarily fit to the final command buffer. Fix
this using a dynamic sbuf buffer. Add a few regression tests for the case.
PR: bin/95079
No objections: freebsd-hackers
Do by specifying ".../" with '-m' or MAKESYSPATH (new) environment variable.
Reviewed by: <sjg@NetBSD.org>
Obtained from: NetBSD (+ embellishment by me, sent back to NetBSD)
- correctly handle error output in $(builtin 2>&1), clarify out1/out2 vs
output/errout in the code
- treat all builtins as regular builtins so errors do not abort the shell
and variable assignments do not persist
- respect the caller's INTOFF
Some bugs still exist:
- expansion errors may still abort the shell
- some side effects of expansions and builtins persist
r195175. Remove all definitions, documentation, and usage.
fifo_misc.c:
Remove all kqueue tests as fifo_io.c performs all those that
would have remained.
Reviewed by: rwatson
MFC after: 3 weeks
X-MFC note: don't change vlan_link_state() function signature
Fix some wrong usages.
Note: this does not affect generated binaries as this argument is not used.
PR: 137213
Submitted by: Eygene Ryabinkin (initial version)
MFC after: 1 month
Reset the exception handler in the child to main's.
This avoids inappropriate double cleanups or shell duplication when the
exception is caught, such as 'fc' and future 'command eval' and 'command .'.
tools/regression. It tests a number of aspects of kqueue behavior,
although not all currently pass (possibly bugs in the test suite?).
Submitted by: Mark Heily <mark at heily.com>
Obtained from: svn://mark.heily.com/libkqueue/trunk/test (r114)
to wcscoll(3). Newline characters could cause incorrect results when
comparing lines.
Also, if an input line didn't contain a newline character, it was
omitted from the output. According to my interpretation, SUSv3 requires
that the newline is always printed.
Add regression tests for the cases. [1]
PR: bin/140976
Submitted by: D'Arcy Cain (original version) [1]
Approved by: trasz (mentor)
**environ entries. This puts non-getenv(3) operations in line with
getenv(3) in that bad environ entries do not cause all operations to
fail. There is still some inconsistency in that getenv(3) in the
absence of any environment-modifying operation does not emit corrupt
environ entry warnings.
I also fixed another inconsistency in getenv(3) where updating the
global environ pointer would not be reflected in the return values.
It would have taken an intermediary setenv(3)/putenv(3)/unsetenv(3)
in order to see the change.
- Redirecting fds that were not open before kept two copies of the
redirected file.
sh -c '{ :; } 7>/dev/null; fstat -p $$; true'
(both fd 7 and 10 remained open)
- File descriptors used to restore things after redirection were not
set close-on-exec, instead they were explicitly closed before executing
a program normally and before executing a shell procedure. The latter
must remain but the former is replaced by close-on-exec.
sh -c 'exec 7</; { exec fstat -p $$; } 7>/dev/null; true'
(fd 10 remained open)
The examples above are simpler than the testsuite because I do not want to
use fstat or procstat in the testsuite.
This avoids weirdness when 'fc -e vi' or the like is done and there is a
syntax error in the file. Formerly an interactive shell tried to execute
stuff after the syntax error and exited.
This should also avoid similar issues with 'command eval' and 'command .'
when 'command' is implemented properly as in NetBSD sh.
Special builtins did not have this problem since errors in them cause the
shell to exit or to reset various state such as the current command input
file.
This seems more useful and will likely be in the next POSIX standard.
Also document more precisely in the man page what set -u does (note that
$@, $* and $! are the only special parameters that can ever be unset, all
the others are always set, although they may be empty).
* retry various system calls on EINTR
* retry the rest after a short read (common if there is more than about 1K
of output)
* block SIGCHLD like system(3) does (note that this does not and cannot
work fully in threaded programs, they will need to be careful with wait
functions)
PR: 90580
MFC after: 1 month
- slightly adjust code for style, sort headers.
- in sigqtest2, print received signals, to make it easy to see why test
failed.
- in sigqtest2, job_control_test(), cover a race by adding sleep after
child stopped itself to allow for SIGCHLD due to stop and exit to not
be coalesced.
MFC after: 2 weeks
This also fixes that trying to execute a non-regular file with a command
name without '/' returns 127 instead of 126.
The fix is rather simplistic: treat CMDUNKNOWN as if the command were found
as an external program. The resulting fork is a bit wasteful but executing
unknown commands should not be very frequent.
PR: bin/137659
Due to the amount of code removed by this, it seems that allowing unmatched
quotes was a deliberate imitation of System V sh and real ksh. Most other
shells do not allow unmatched quotes (e.g. bash, zsh, pdksh, NetBSD /bin/sh,
dash).
PR: bin/137657
The most important test is the mapping fixed at address 0 depending on the
new sysctl.
Things will be updated and possibly converted to m4/.t style once the
details about the kernel patch will be shaken out.
Submitted by: simon (initial version)
"The escape sequence '\n' shall match a <newline> embedded in
the pattern space."
It is unclear whether this also applies to a \n embedded in a
character class. Disable the existing handling of \n in a character
class following Mac OS X, GNU sed version 4.1.5 with --posix, and
SunOS 5.10 /usr/bin/sed.
Pointed by: Marius Strobl
Obtained from: Mac OS X
of the y (translate) command.
"If a backslash character is immediately followed by a backslash
character in string1 or string2, the two backslash characters shall
be counted as a single literal backslash character"
Pointed by: Marius Strobl
Obtained from: Mac OS X
Empty pairs of braces are represented by a NULL node pointer, just like
empty lines at the top level.
Support for empty pairs of braces may be removed later. They make the code
more complex, have inconsistent behaviour (may or may not change $?), are
not specified by POSIX and are not allowed by some other shells like bash,
dash and ksh93.
Reported by: kan
Add a reference count to function definitions.
Memory may leak if multiple SIGINTs arrive in interactive mode,
this will be fixed later by changing SIGINT handling.
PR: bin/137640
Make regression/priv compile again after the multi-IP jail
changes. Note that we are still using the legacy jail(2)
rather than the jail_set(2)/jail(3) syscall.
Add an IPv4, and an IPv6 loopback address in case we compile
with INET6 enabled.
Make the priv_vfs_extattr_system compile on amd64 as well using the
proper length modifier to printf(3) for ssize_t.
Reviewed by: rwatson
Approved by: re (kib)
display '+' on them. Taken from kern/125613, with cosmetic
changes.
PR: kern/125613
Submitted by: Jaakko Heinonen <jh at saunalahti dot fi>
Approved by: re (kib)
any open file descriptors >= 'lowfd'. It is largely identical to the same
function on other operating systems such as Solaris, DFly, NetBSD, and
OpenBSD. One difference from other *BSD is that this closefrom() does not
fail with any errors. In practice, while the manpages for NetBSD and
OpenBSD claim that they return EINTR, they ignore internal errors from
close() and never return EINTR. DFly does return EINTR, but for the common
use case (closing fd's prior to execve()), the caller really wants all
fd's closed and returning EINTR just forces callers to call closefrom() in
a loop until it stops failing.
Note that this implementation of closefrom(2) does not make any effort to
resolve userland races with open(2) in other threads. As such, it is not
multithread safe.
Submitted by: rwatson (initial version)
Reviewed by: rwatson
MFC after: 2 weeks
because it means getdelim() returns -1 for both error and EOF, and
never returns 0. However, this is what the original GNU implementation
does, and POSIX inherited the bug.
Reported by: marcus@
colliding upper case letters as the lower case letter with a '_' in
front.
MFC after: 3 days
Discussed with: ed
Spotted by: Michael David Crawford <mdc at prgmr.com>
The function pow() in libmp(3) clashes with pow(3) in libm. We could
rename this single function, but we can just take the same approach as
the Solaris folks did, which is to prefix all function names with mp_.
libmp(3) isn't really popular nowadays. I suspect not a single
application in ports depends on it. There's still a chance, so I've
increased the SHLIB_MAJOR and __FreeBSD_version.
Reviewed by: deischen, rdivacky
It is only really necessary for open(2)'s third argument, which is optional and
obtained through stdarg(3). open(2)'s third argument is 32bit and we pass 64
bits. On little endian it works, because we take lower 32 bits, but on big
endian platforms we take upper 32 bits, so we end up with 0.
Reported by: Milan Čermák <Milan.Cermak@Sun.COM>
allocated in a fork(2)-inheritable way at the beginning or end of an
accept(2) system call. This test creates a test thread and blocks it
in accept(2), then forks a child process which tests to see if the
next available file descriptor is defined or not (EBADF vs EINVAL for
ftruncate(2)).
This detects a regression introduced during the network stack locking
work, in which a very narrow race during which fork(2) from one
thread during accept(2) in a second thread lead to an extra inherited
file descriptor turned into a very wide race ensuring that a
descriptor was leaked into the child even though it hadn't been
returned.
PR: kern/130348
The jot(1) regression tests directory contained two tests named `wx' and
`wX', which doesn't work on case insensitive filesystems. Rename `wX' to
`wX1'.
MFC after: 1 month
This bring huge amount of changes, I'll enumerate only user-visible changes:
- Delegated Administration
Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.
- L2ARC
Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.
- slog
Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).
- vfs.zfs.super_owner
Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.
- chflags(2)
Not all the flags are supported. This still needs work.
- ZFSBoot
Support to boot off of ZFS pool. Not finished, AFAIK.
Submitted by: dfr
- Snapshot properties
- New failure modes
Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests
- Refquota, refreservation properties
Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.
- Sparse volumes
ZVOLs that don't reserve space in the pool.
- External attributes
Compatible with extattr(2).
- NFSv4-ACLs
Not sure about the status, might not be complete yet.
Submitted by: trasz
- Creation-time properties
- Regression tests for zpool(8) command.
Obtained from: OpenSolaris
"A function can be preceded by one or more '!' characters, in which
case the function shall be applied if the addresses do not select
the pattern space."
from one parent directory to another, in addition to the usual access checks
one also needs write access to the subdirectory being moved.
Approved by: rwatson (mentor), pjd
and server. This replaces the RPC implementation of the NFS client and
server with the newer RPC implementation originally developed
(actually ported from the userland sunrpc code) to support the NFS
Lock Manager. I have tested this code extensively and I believe it is
stable and that performance is at least equal to the legacy RPC
implementation.
The NFS code currently contains support for both the new RPC
implementation and the older legacy implementation inherited from the
original NFS codebase. The default is to use the new implementation -
add the NFS_LEGACYRPC option to fall back to the old code. When I
merge this support back to RELENG_7, I will probably change this so
that users have to 'opt in' to get the new code.
To use RPCSEC_GSS on either client or server, you must build a kernel
which includes the KGSSAPI option and the crypto device. On the
userland side, you must build at least a new libc, mountd, mount_nfs
and gssd. You must install new versions of /etc/rc.d/gssd and
/etc/rc.d/nfsd and add 'gssd_enable=YES' to /etc/rc.conf.
As long as gssd is running, you should be able to mount an NFS
filesystem from a server that requires RPCSEC_GSS authentication. The
mount itself can happen without any kerberos credentials but all
access to the filesystem will be denied unless the accessing user has
a valid ticket file in the standard place (/tmp/krb5cc_<uid>). There
is currently no support for situations where the ticket file is in a
different place, such as when the user logged in via SSH and has
delegated credentials from that login. This restriction is also
present in Solaris and Linux. In theory, we could improve this in
future, possibly using Brooks Davis' implementation of variant
symlinks.
Supporting RPCSEC_GSS on a server is nearly as simple. You must create
service creds for the server in the form 'nfs/<fqdn>@<REALM>' and
install them in /etc/krb5.keytab. The standard heimdal utility ktutil
makes this fairly easy. After the service creds have been created, you
can add a '-sec=krb5' option to /etc/exports and restart both mountd
and nfsd.
The only other difference an administrator should notice is that nfsd
doesn't fork to create service threads any more. In normal operation,
there will be two nfsd processes, one in userland waiting for TCP
connections and one in the kernel handling requests. The latter
process will create as many kthreads as required - these should be
visible via 'top -H'. The code has some support for varying the number
of service threads according to load but initially at least, nfsd uses
a fixed number of threads according to the value supplied to its '-n'
option.
Sponsored by: Isilon Systems
MFC after: 1 month
it relies on non-portable flock(2) semantics. Not only is flock(2) not
portable, but on some OSes that do have it, it is implemented in terms
of fcntl(2) locks, which are per-process rather than per-descriptor.
will cause it to return 0, not EAGAIN.
Add UNIX domain socket support to udpzerobyte, which suggests this
regression test should be moved to the general sockets test area rather
than netinet.
once it is lost, all data is gone.
Option '-B none' can by used to prevent backup. Option '-B path' can be
used to backup metadata to a different file than the default, which is
/var/backups/<prov>.eli.
The 'geli init' command also prints backup file location and gives short
procedure how to restore metadata.
The 'geli setkey' command now warns that even after passphrase change or keys
update there could be version of the master key encrypted with old
keys/passphrase in the backup file.
Add regression tests to verify that new functionality works as expected.
Update other regression tests so they don't create backup files.
Reviewed by: keramida, rink
Dedicated to: a friend who lost 400GB of his live by accidentally overwritting geli metadata
MFC after: 2 weeks
the first value (environ[0]) to NULL. This is in addition to the
current detection of environ being replaced, which includes being set to
NULL. Without this fix, the environment is not truly wiped, but appears
to be by getenv() until an *env() call is made to alter the enviroment.
This change is necessary to support those applications that use this
method for clearing environ such as Dovecot and Postfix. Applications
such as Sendmail and the base system's env replace environ (already
detected). While neither of these methods are defined by SUSv3, it is
best to support them due to historic reasons and in lieu of a clean,
defined method.
Add extra units tests for clearing environ using four different methods:
1. Set environ to NULL pointer.
2. Set environ[0] to NULL pointer.
3. Set environ to calloc()'d NULL-terminated array.
4. Set environ to static NULL-terminated array.
Noticed by: Timo Sirainen
MFC after: 3 days
the default ICMPv6 filter is pass all, test that we can set it to block
all and restore to pass all. No attempt is made to test that the
filtering works, just that we can get and set it.
I wrote these to test amd64 asm functions that used
maxss, maxsd, minss, and minsd, but it turns out that
those instructions don't handle NaNs and signed zero
in the same way as fmin() and fmax() are required to,
so we're stuck with the C versions for now.
The first test comes from OpenBSD, and the others are additions or
adaptations.
This is based on OpenBSD's
src/regress/lib/libc/sprintf/sprintf_test.c, v1.3.
I deliberately did not use v1.4 because it's bogus.
semaphores. Specifically, semaphores are now represented as new file
descriptor type that is set to close on exec. This removes the need for
all of the manual process reference counting (and fork, exec, and exit
event handlers) as the normal file descriptor operations handle all of
that for us nicely. It is also suggested as one possible implementation
in the spec and at least one other OS (OS X) uses this approach.
Some bugs that were fixed as a result include:
- References to a named semaphore whose name is removed still work after
the sem_unlink() operation. Prior to this patch, if a semaphore's name
was removed, valid handles from sem_open() would get EINVAL errors from
sem_getvalue(), sem_post(), etc. This fixes that.
- Unnamed semaphores created with sem_init() were not cleaned up when a
process exited or exec'd. They were only cleaned up if the process
did an explicit sem_destroy(). This could result in a leak of semaphore
objects that could never be cleaned up.
- On the other hand, if another process guessed the id (kernel pointer to
'struct ksem' of an unnamed semaphore (created via sem_init)) and had
write access to the semaphore based on UID/GID checks, then that other
process could manipulate the semaphore via sem_destroy(), sem_post(),
sem_wait(), etc.
- As part of the permission check (UID/GID), the umask of the proces
creating the semaphore was not honored. Thus if your umask denied group
read/write access but the explicit mode in the sem_init() call allowed
it, the semaphore would be readable/writable by other users in the
same group, for example. This includes access via the previous bug.
- If the module refused to unload because there were active semaphores,
then it might have deregistered one or more of the semaphore system
calls before it noticed that there was a problem. I'm not sure if
this actually happened as the order that modules are discovered by the
kernel linker depends on how the actual .ko file is linked. One can
make the order deterministic by using a single module with a mod_event
handler that explicitly registers syscalls (and deregisters during
unload after any checks). This also fixes a race where even if the
sem_module unloaded first it would have destroyed locks that the
syscalls might be trying to access if they are still executing when
they are unloaded.
XXX: By the way, deregistering system calls doesn't do any blocking
to drain any threads from the calls.
- Some minor fixes to errno values on error. For example, sem_init()
isn't documented to return ENFILE or EMFILE if we run out of semaphores
the way that sem_open() can. Instead, it should return ENOSPC in that
case.
Other changes:
- Kernel semaphores now use a hash table to manage the namespace of
named semaphores nearly in a similar fashion to the POSIX shared memory
object file descriptors. Kernel semaphores can now also have names
longer than 14 chars (up to MAXPATHLEN) and can include subdirectories
in their pathname.
- The UID/GID permission checks for access to a named semaphore are now
done via vaccess() rather than a home-rolled set of checks.
- Now that kernel semaphores have an associated file object, the various
MAC checks for POSIX semaphores accept both a file credential and an
active credential. There is also a new posixsem_check_stat() since it
is possible to fstat() a semaphore file descriptor.
- A small set of regression tests (using the ksem API directly) is present
in src/tools/regression/posixsem.
Reported by: kris (1)
Tested by: kris
Reviewed by: rwatson (lightly)
MFC after: 1 month
provides the correct semantics for flock(2) style locks which are used by the
lockf(1) command line tool and the pidfile(3) library. It also implements
recovery from server restarts and ensures that dirty cache blocks are written
to the server before obtaining locks (allowing multiple clients to use file
locking to safely share data).
Sponsored by: Isilon Systems
PR: 94256
MFC after: 2 weeks
fifos, as this is required by the Single UNIX Specification, although
not currently implemented on FreeBSD.
While here, fix a bug in the directory timestamp checking test by
sleeping after querying the starting timestamp, rather than before.
AIO calls.
This small program queues up a controllable number of concurrent AIO
read operations w/ controllable io size against a disk or regular file.
There are a few other things to add (notably optional write support!)
but it works well enough at the present time to stress the AIO code out
relatively harshly in the disk IO case.
after similar calls related to struct pwd in libutil/pw_util.c:
- gr_equal()
Perform a deep comparison of two struct grp's. It does a thorough, yet
unoptimized comparison of all the members regardless of order.
- gr_make()
Create a string (see group(5)) from a struct grp.
- gr_dup()
Duplicate a struct grp. Returns a value that is a single contiguous
block of memory.
- gr_scan()
Create a struct grp from a string (as produced by gr_make()).
MFC after: 3 weeks
Solaris and AIX.
fcntl(fd, F_DUP2FD, arg) and dup2(fd, arg) are functionnaly equivalent.
Document it.
Add some regression tests (identical to the dup2(2) regression tests).
PR: 120233
Submitted by: Jukka Ukkonen
Approved by: rwaston (mentor)
MFC after: 1 month
Use the correct value of errno. Although the errno value passed into
printf() follows the *env() call, it is not guaranteed to be the errno
from that call. When I wrote the regression tester, the environment I
used did pass the errno from the call. Consolidate the print for the
return code and errno into a function in the process of fixing this.
Approved by: wes (mentor)
various open flags and then tests various operations to make sure that
they are properly constrained by open flags. Various I/O mechansms
are tried, including aio if compiled into the kernel or loaded as a
module. There's more to be done here but it's a useful start, running
about 220 individual tests.
This is in support of FreeBSD-SA-08:03.sendfile.
the semantics of pthread_mutex_islocked_np() to return true if and only if
the mutex is held by the current thread.
Obviously, change the regression test to match.
MFC after: 2 weeks
