The daemonfd function is equivalent to the daemon(3) function expect that
arguments are descriptors. For example dhclient(8) which is sandboxed is
unable to open /dev/null to close stdio instead it's allows to fail
daemon(3) function to close the descriptors and then do it explicit in code.
Instead of such hacks we can use now daemonfd.
This API can be also helpful to migrate system to platforms like CheriBSD.
Reviewed by: brooks@, bcr@, jilles@ (earlier version)
Differential Revision: https://reviews.freebsd.org/D13433
There are two versions of variant I of TLS
- ARM and aarch64 uses original version of variant I here TP points to
start of TCB followed by aligned TLS segment. Both TCB and TLS must
be aligned to alignment of TLS section. The TCB[0] points to DTV vector
and DTV values are real addresses (without bias).
- MIPS, PowerPC and RISC-V use modified version of variant I,
where TP points (with bias) to TLS and TCB immediately precedes TLS
without any alignment gap. Only TLS should be aligned. The TCB[0]
points to DTV vector and DTV values are biased by constant value (0x8000)
from real addresses.
Take all this in account when allocating memory for TLS structures.
MFC after: 1 month
Reviewed by: kib, mizhka
Tested by: mizhka(on mips)
Differential Revision: https://reviews.freebsd.org/D13378
Now that the POSIX working group is going to require that basename(3)
and dirname(3) are thread-safe in future revisions of the standard,
there is even less of a need to provide basename_r(3). Remove this
function to prevent people from writing code that only builds on
FreeBSD and Bionic.
Removing this function seems to break exactly one port: sbruno@'s
qemu-user-static. I will send him a pull request on GitHub in a bit.
__FreeBSD_version will not be bumped, as any value from 2017 can be used
to test for the presence of a thread-safe basename(3)/dirname(3).
PR: https://bugs.freebsd.org/224016
Currently each call to telldir() requires a malloc and adds an entry to a
linked list which must be traversed on future telldir(), seekdir(),
closedir(), and readdir() calls. Applications that call telldir() for every
directory entry incur O(n^2) behavior in readdir() and O(n) in telldir() and
closedir().
This optimization eliminates the malloc() and linked list in most cases by
packing the relevant information into a single long. On 64-bit architectures
msdosfs, NFS, tmpfs, UFS, and ZFS can all use the packed representation. On
32-bit architectures msdosfs, NFS, and UFS can use the packed
representation, but ZFS and tmpfs can only use it for about the first 128
files per directory. Memory savings is about 50 bytes per telldir(3) call.
Speedup for telldir()-heavy directory traversals is about 20-30x for one
million files per directory.
Reviewed by: kib, mav, mckusick
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13385
system calls. Man pages are missing for v2 and v5, so any entries for
those versions were inferred by new implementations of these functions
in libc.
Obtained from: http://www.tuhs.org/cgi-bin/utree.pl
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
For statically linked binaries, where all relocation are solved by static
linker, the linker expect that offset to TLS section is aligned. Additionaly,
to maintain absolute alignment, TLS TCB should by also aligned.
Obtained from: CheriBSD (initial version)
MFC after: 1 month
Reviewed by: brooks (previous version), kib
Differential Revision: https://reviews.freebsd.org/D12907
one call to sysctl(2) from jemalloc startup code. (That also requires
changes to jemalloc, but I plan to push those to upstream first.)
Reviewed by: kib
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12745
- Teach elf aux vector functions about newly added AT_HWCAP and AT_HWCAP2
vectors.
- Export _elf_aux_info() as new public libc function elf_aux_info(3)
The elf_aux_info(3) should be considered as FreeBSD counterpart of glibc
getauxval() with more robust interface.
Note:
We cannot name this new function as getauxval(), with glibc compatible
interface. Some ports autodetect its existence and then expects that all
Linux specific AT_<*> vectors are defined and implemented.
MFC after: 1 month
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D12743
On Variant I TLS architectures (aarch64, arm, mips, powerpc, and riscv)
the __libc_allocate_tls function allocates thread local storage memory
with calloc(). It then copies initialization data over the portions with
non-zero initial values. Before this change it would then pointlessly
zero the already zeroed remainder of the storage. Unfortunately the
calculation was wrong and it would zero TLS_TCB_SIZE (2*sizeof(void *))
additional bytes.
In practice, this overflow only matters if the TLS segment is sized such
that calloc() allocates a less than TLS_TCB_SIZE extra memory. Even
then, the likely result will be zeroing part of the next bucket. This
coupled with the impact being confined to Tier II platforms means there
will be no security advisory for this issue.
Reviewed by: kib, dfr
Discussed with: security-officer (delphij)
MFC after: 1 week
Found by: CHERI
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12547
Illumos and Schillix is adopting some of the locale code and our style(9)
sometimes matches the Solaris cstyle, so the changes are also useful as a
way to reduce diffs.
No functional change.
Discussed with: Joerg Schilling
MFC after: 1 week
getmntinfo(3) is designed around a relatively static or slow growing set of
current mounts. It tried to detect a race with somewhat concurrent mount
and re-call getfsstat(2) in that case, looping indefinitely. It also
allocated space for a single extra mount as slop.
In the case where the user has a large number of mounts and is adding them
at a rapid pace, it fell over.
This patch makes two functional changes:
1. Allocate even more slop. Double whatever the last getfsstat(2) returned.
2. Abort and return some known results after looping a few times
(arbitrarily, 3). If the list is constantly changing, we can't guarantee
we return a full result to the user at any point anyways.
While here, add very basic functional tests for getmntinfo(3) to the libc
suite.
PR: 221743
Submitted by: Peter Eriksson <peter AT ifm.liu.se> (earlier version)
Sponsored by: Dell EMC Isilon
In a recent interpretation[1], "\\" shall return a non-zero value
(indicating either no match or an error).
The fix involves a change over r254091 and now the behavior matches the
Sun/IBM/HP closed source implementations and also likely musl libc.
Submitted by: Joerg Schilling <joerg at schily.net>
MFC after: 1 week
[1] http://austingroupbugs.net/view.php?id=806
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)
This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.
RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):
__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen
Reviewed by: ngie
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D11901
Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions. The functions are resolved from the rtld instead, the goal
of creating library is to avoid errors from the static linker due to
missed libdl. For static binaries, an empty .o is compiled into
libdl.a so that static binaries still get dl stubs from libc.a.
Right now lld cannot create filter objects, disable libdl on arm64
when binutils are not used.
Reviewed by: bdrewery, dim (previos version); emaste
Exp run: PR 220525, done by antoine
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D11504
Most important, use a correct signature for the
__pthread_cleanup_push_imp() stub, which was incorrectly generated
with two-args variant. The pthread_cleanup_info pointer was corrupted
in the forwarded call to the real libthr implementation, visible on
PowerPC and possibly ARM. [1]
Found and tested by: Mark Millard <markmi@dsl-only.net> [1]
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Make syslog(3) resilent to cancellation occuring in supported deferred
mode. Code must unlock syslog_mutex on cancel, install the cleanup
handler.
Diagnosed and tested by: eugen
Discussed with: dchagin
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
gets output via warnx(3)
This helps set expectations for how one might deal with those messages, i.e.,
mute output from /dev/stderr today, since that's where vwarn(3) outputs messages
to today.
MFC after: 1 month
Move an unneeded initialization, introduced in r54770 to quiet down GCC,
to a place nearer to its first use. This has no practical effect, it just
keeps the garbage better sorted.
Hinted by: OpenBSD (CVS rev. 1.56, without obfuscations)
Extend the ino_t, dev_t, nlink_t types to 64-bit ints. Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment. Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.
ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks. Unfortunately, not everything can be
fixed, especially outside the base system. For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.
Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.
Struct xvnode changed layout, no compat shims are provided.
For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.
Update note: strictly follow the instructions in UPDATING. Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.
Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver. Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem). Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).
Sponsored by: The FreeBSD Foundation (emaste, kib)
Differential revision: https://reviews.freebsd.org/D10439
This no longer seems useful. Remove it.
This was prompted by a "cast discards volatile qualifier" warning
in libthr when WARNS=6.
Reviewed by: kib
MFC after: 3 days
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10832
_local_initshells did not reset cp to the beginning of the line buffer
for every iteration that it called fgets(3), leading to writing past the
end of line with fairly long /etc/shells or excessively long line
lengths. Correct this by properly resetting cp.
PR: 192528
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: cem, jilles
Differential Revision: https://reviews.freebsd.org/D10690
Trivial style(9) fix, no functional change. There are also some 81
characters lines below, but I don't see a good way to shorten them.
Sponsored by: Dell EMC Isilon
Adapt glob's match() routine to use a greedy algorithm that avoids
exponential runtime in byzantine inputs.
While here, add a testcase for the byzantine input.
Prompted by: https://research.swtch.com/glob
Authored by: Yves Orton <demerphq at gmail.com>
Obtained from: Perl (33252c318625f3c6c89b816ee88481940e3e6f95)
Sponsored by: Dell EMC Isilon
The mutex is used in sem_open() and sem_close(), which cannot
recurse. The atfork handlers cannot collide with the open and close
code.
Reviewed by: vangyzen
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D10545
Do not retest for the found semaphore after the loop to look it up.
Instead, handle both cases of last and non-last close simultaneously,
which allows to consolidate the list unlock and successful return.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
The sysctl(HW_PAGESIZE) call cannot fail on FreeBSD kernels at least.
And even if it failed for some improbable reason, PAGE_SIZE is a safe
value to return.
Discussed with: jilles
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
The internal array size goes through a loop and is compared with numitems
which at its limits makes can be unreachably higher than arraysz.
Prevent an hypothetical overflow by matching the types.
MFC after: 1 week
When application reads large directory, calling telldir() for each entry,
like Samba does, it creates exponential performance drop as number of
entries reach tenths to hundreds of thousands. It is caused by full search
through the internal list, that never finds matches in that scenario, but
creates O(n^2) delays. This patch optimizes that search, limiting it to
entries of the same buffer, turning time closer to O(n) in case of linear
directory scan.
PR: 218622
Reviewed by: jhb, jilles
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D10408
of the current usermode implementation of the POSIX semaphores.
For NSEMS_MAX, return -1 without changing errno, which indicates that
the variable has no limit. Before, sysconf(3) returned parameters
queried from the ksem(9) legacy implementation, which apparently has
low defaults for NSEMS_MAX.
Reported and tested by: jbeich
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Implement a new init(8) option in /etc/ttys. If this option is present
on the entry in /etc/ttys, the entry will be active if and only if it
exists. If the name starts with a '/', it will be considered an
absolute path. If not, it will be a path relative to /dev.
This allows one to turn off video console getty that aren't present
(while running a getty on them even when they aren't the system
console). Likewise with serial ports.
It differs from onifconsole in only requiring the device exist rather
than it be listed as one of the system consoles.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D10037
If opendir succeeds but malloc fails, numitems was used uninitialized in
error handling under the 'fail' label. If it happened to have a non-zero
value, the NULL 'names' was dereferenced.
Reported by: Coverity
CIDs: 1329566, 1372625
Sponsored by: Dell EMC Isilon
reallocarray(3) is a non portable extension that originated in OpenBSD.
Given that it is already in FreeBSD's libc it is useful for the cases
where reallocation involves a multiplication.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D9955
Unsign setlen: it is local and will never be negative. Having one more bit
for growth is beneficial and it avoids a cast when it's going to be used
for allocation.
Reviewed by: ngie
MFC after: 3 days
Rename nitems to numitems: it shares the anme with an existing macro in
sys/params.h. Also initialize the value later which avoids asigning the
value if we exit early.
Reviewed by: ngie
MFC after: 3 days
Dan Krejsa reports a potential memory leak in an fts_build error case,
detected by Coverity. (It doesn't seem to show up in Coverity Scan, so I
don't have a CID to point to.)
I don't know whether it is actually possible to arrive in this case with a
non-empty 'head' list. The cost is low, though. One additional branch in a
terminal error case isn't the end of the world.
PR: 217125
Submitted by: Dan Krejsa <dan.krejsa at gmail.com>
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
Also mention <time.h> in sem_timedwait(3), because POSIX does,
and because the user will need it for clockid_t, struct timespec,
and TIMER_ABSTIME.
Reported by: bde
MFC after: 9 days
X-MFC with: r314179
Sponsored by: Dell EMC
This function allows the caller to specify the reference clock
and choose between absolute and relative mode. In relative mode,
the remaining time can be returned.
The API is similar to clock_nanosleep(3). Thanks to Ed Schouten
for that suggestion.
While I'm here, reduce the sleep time in the semaphore "child"
test to greatly reduce its runtime. Also add a reasonable timeout.
Reviewed by: ed (userland)
MFC after: 2 weeks
Relnotes: yes
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D9656
This reduces build output, need for recalculating paths, and makes it clearer
which paths are relative to what areas in the source tree. The change in
performance over a locally mounted UFS filesystem was negligible in my testing,
but this may more positively impact other filesystems like NFS.
LIBC_SRCTOP was left alone so Juniper (and other users) can continue to
manipulate lib/libc/Makefile (and other Makefile.inc's under lib/libc) as
include Makefiles with custom options.
Discussed with: marcel, sjg
MFC after: 1 week
Reviewed by: emaste
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D9207
drain timeout handling to historical freebsd behavior.
The primary reason for these changes is the need to have tty_drain() call
ttydevsw_busy() at some reasonable sub-second rate, to poll hardware that
doesn't signal an interrupt when the transmit shift register becomes empty
(which includes virtually all USB serial hardware). Such hardware hangs
in a ttyout wait, because it never gets an opportunity to trigger a wakeup
from the sleep in tty_drain() by calling ttydisc_getc() again, after
handing the last of the buffered data to the hardware.
While researching the history of changes to tty_drain() I stumbled across
some email describing the historical BSD behavior of tcdrain() and close()
on serial ports, and the ability of comcontrol(1) to control timeout
behavior. Using that and some advice from Bruce Evans as a guide, I've
put together these changes to implement the hardware polling and restore
the historical timeout behaviors...
- tty_drain() now calls ttydevsw_busy() in a loop at 10 Hz to accomodate
hardware that requires polling for busy state.
- The "new historical" behavior for draining during close(2) is retained:
the drain timeout is "1 second without making any progress". When the
1-second timeout expires, if the count of bytes remaining in the tty
layer buffer is smaller than last time, the timeout is extended for
another second. Unfortunately, the same logic cannot be extended all
the way down to the hardware, because the interface to that layer is a
simple busy/not-busy indication.
- Due to the previous point, an application that needs a guarantee that
all data has been transmitted must use TIOCDRAIN/tcdrain(3) before
calling close(2).
- The historical behavior of honoring the drainwait setting for TIOCDRAIN
(used by tcdrain(3)) is restored.
- The historical kern.drainwait sysctl to control the global default
drainwait time is restored, but is now named kern.tty_drainwait.
- The historical default drainwait timeout of 300 seconds is restored.
- Handling of TIOCGDRAINWAIT and TIOCSDRAINWAIT ioctls is restored
(this also makes the comcontrol(1) drainwait verb work again).
- Manpages are updated to document these behaviors.
Reviewed by: bde (prior version)
This argument is not a bitmask of flags, but only accepts a single value.
Fail with EINVAL if an invalid value is passed to 'flag'. Rename the
'flags' argument to getmntinfo(3) to 'mode' as well to match.
This is a followup to r308088.
Reviewed by: kib
MFC after: 1 month
These functions are supposed to return a value between [_2^31, 2^31).
This doesn't seem to work on 64-bit systems, where we return a value
between [0, 3^32). Patch up the function to use proper casts to int32_t.
While there, fix some other style bugs.
MFC after: 2 weeks
Now that the changes to the dirname(3) function had some time to settle,
let's go ahead and use the same approach for replacing basename(3) by a
simple implementation that modifies the input string, thereby making it
thread-safe and guaranteed to succeed.
Unlike dirname(3), this function already had a thread-safe variant
basename_r(3). This function had its own set of problems, like having an
upper bound on the pathname length. Keep this function around for
compatibility, but remove most references from the man page. Make the
man page more similar to that of dirname(3).
As the basename_r(3) function is only provided by FreeBSD (and Bionic),
depending on its use is even more implementation defined than assuming
that basename(3) is thread-safe.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D8382
Specifically, use .Ta instead of tabs to separate column entries. While
here fix a few other things:
- Use .Sy for all column headers (previously only the first column header
was bold)
- Use .Dv to markup constants used for MIB names.
- Use "1234" and "4321" for the byte order descriptions without
thousands separators.
- Mark up header files in the first table with .In.
MFC after: 2 weeks
The setkey() and encrypt() functions are part of XSI, not the POSIX base
definitions. There is no strict requirement for us to provide these,
especially if we're only going to keep these around as undocumented
stubs. The same holds for des_setkey() and des_cipher().
Instead of providing functions that only generate warnings when linking,
simply disallow linking against them. The impact of this is relatively
low. It only causes two leaf ports to break. I'll see what I can do to
help out to get those fixed.
PR: 211626
The sysctl cannot fail. If it does fail on some FreeBSD derivative or
after some future change, just abort() so that the problem will be found
and fixed.
It's preferable to provide an arc4random() function that cannot fail and
cannot return poor quality random data. While abort() is not normally
suitable for a library, it makes sense here.
Reviewed by: ed, jonathan, markm
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8077
It turns out that the path normalization that our brand new copy of
dirname(3) does is actually not allowed by the draft version of the
upcoming version of POSIX. It has to behave identically to the
dirname(1) utility.
This change replaces our new dirname(3) implementation by yet another
version that doesn't implement the path normalization logic; it merely
looks for the end of the directory name and overwrites that with a null
byte.
More details: See note #3370 at http://austingroupbugs.net/view.php?id=1073
PR: 212193
Reviewed by: emaste, jilles
Differential Revision: https://reviews.freebsd.org/D7790
In existing implementations including FreeBSD, there is no reason to use
readdir_r() in the common case where potentially multiple threads each list
their own directory. Code using readdir() is simpler.
What's more, lthough readdir_r() can safely be used on FreeBSD because
NAME_MAX is forced to 255, it cannot be used safely on systems where
{NAME_MAX} is not fixed. As a concrete example, FAT/NTFS filenames can be up
to 255 UTF-16 code units long, which can be up to 765 UTF-8 bytes.
Deprecating readdir_r() in POSIX has been proposed in
http://www.austingroupbugs.net/view.php?id=696
and glibc wants to deprecate it as well.
Reviewed by: ed, wblock
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D7678
As the xinstall(8) utility had to be patched up to work with the POSIXly
correct basename()/dirname() prototypes, we make it pretty hard to build
previous versions of FreeBSD on HEAD. xinstall(8) is part of the
bootstrap tools.
Add some logic to <libgen.h> to automatically detect bad calls to
dirname() based on the type of the argument. If the argument is of type
'const char *', we simply fall back to calling into dirname@FBSD_1.0
directly.
I'll also give basename() similar treatment when importing the
thread-safe version of that function.
Tested by: bdrewery, madpilot (thanks!)
Now that we've updated the prototypes of the basename(3) and dirname(3)
functions to conform to POSIX, let's go ahead and reimplement dirname(3)
in such a way that it's thread-safe, but also guaranteed to succeed. C
libraries like glibc, musl and the one that's part of Solaris already
follow such an approach.
Move the existing implementation to another source file,
freebsd11_dirname.c to keep existing users of the API that pass in a
constant string happy, using symbol versioning.
Put a new version of the function in dirname.c, obtained from CloudABI's
C library. This version scans through the pathname string from left to
right, normalizing it, while discarding the last pathname component.
Reviewed by: emaste, jilles
Differential Revision: https://reviews.freebsd.org/D7355
two sepatate functions to make glob(3) code less obscure and more simple.
There is no needs to make them inline since it is error path which supposed
to not happes often.
Just like with freelocale(3), I haven't been able to find any piece of
code that actually makes use of this function's return value, both in
base and in ports. The reason for this is that FreeBSD seems to be the
only operating system to have such a prototype. This is why I'm deciding
to not use symbol versioning for this.
It does seem that the pw(8) utility depends on the function's typing and
already had a switch in place to toggle between the FreeBSD and POSIX
variant of this function. Clean this up by always expecting the POSIX
variant.
There is also a single port that has a couple of local declarations of
setgrent(3) that need to be patched up. This is in the process of being
fixed.
PR: 211394 (exp-run)
to 0. Breaking this rule in 2001 NetBSD hack was imported which attempts
to workaround very limited glob() return codes amount. Use POSIX-compatible
workaround now with E2BIG which can't comes from other functions used
instead of prohibited 0.
Update the existing manual pages for basename(3) and dirname(3) to
mention that in future versions of FreeBSD, these functions will no
longer use internal buffers for storing the results.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D7356
POSIX allows these functions to be implemented in a way that the
resulting string is stored in the input buffer. Though some may find
this annoying, this has the advantage that it makes it possible to
implement this function in a thread-safe way. It also means that they
can be implemented in a way that they work for paths of arbitrary
length, as the output string of these functions is never longer than
max(1, len(input)).
Portable code already needs to be written with this in mind, so in my
opinion it makes very little sense to allow the existing behaviour.
Prevent the base system from falling back to this by switching over to
POSIX prototypes.
I'm not going to bump the __FreeBSD_version for this. The reason is that
it's possible to account for this change in a portable way, without
depending on a specific version of FreeBSD. An exp-run was done some
time ago. As far as I know, all regressions as a result of this have
already been fixed.
I'll give this change some time to settle. In the long run I want to
replace our copies by ones that are thread-safe and don't depend on
PATH_MAX/MAXPATHLEN.
case it fully constructed, but for half-constructed too, so have no
other choice to pass original pattern from glob() down to globextend()
instead of attempt to reconstruct I implement previously.
2) Instead of copy&paste the same big enough code, make function for it:
globfinal().
globexp1() recursive calls, but glob0() was not supposed to be called
repeatedly in the original code. It finalize results by possible adding
original pattern for no match case, may return GLOB_NOMATCH error and
by sorting all things. Original pattern adding or GLOB_NOMATCH error
can happens each time glob0() called repeatedly, and sorting happens
for one item only, all things are never sorted. Second, f.e. "a{a"
pattern does not match "a{a" file but match "a" file instead
(just one example, there are many). Third, some errors (f.e. for limits
or overflow) can be ignored by GLOB_BRACE code because it forces return (0).
Add non-finalizing flag to glob0() and make globexp0() wrapper around
recursively called globexp1() to finalize things like glob0() does.
Reorganize braces code to work correctly.
2) Don't allow MB_CUR_MAX * strlen overallocation hits GLOB_LIMIT_STRING
(ARG_MAX) limit, use final string length, not malloced space for it.
3) Revive DEBUG-ifdefed section.
unmodified, if no matches found. But our original code strips all '\'
returning it. Rewrite the code to allow to reconstruct exact the
original pattern with backslashes for this case.
2) Prevent to use truncated pattern if MAXPATHLEN exceeded, return
GLOB_NOMATCH instead.
3) Fix few end loop conditions filling Char arrays with mbrtowc(),
MB_CUR_MAX is unneeded in two places and condition is less by one
in other place.
4) Prevent to use truncated filenames match if MAXPATHLEN exceeded,
skip such directory entries.
5) Don't end *pathend with L'/' in glob3() if limit is reached, this
change will be not visible since error is returned.
6) If error happens in (*readdirfunc)(), do the same GLOB_ABORTED
processing as for g_opendir() as POSIX requires.
in any case and needed for further processing. For ~ expansion too.
2) Don't terminate *pathend with / when GLOB_LIMIT_STAT is reached, it will
be not visible outside in any case since error is returned.
3) Cosmetic: change if expression to better reflect its semantic.
since whole conversion needs a room for (len >= MB_CUR_MAX). It is no
difference when MB_CUR_MAX == 1, but for multi-byte locales last few chars
('\0' and before) may need just one byte, and the rest of MB_CUR_MAX - 1
space becomes unavailable in the MAXPATHLEN-sized buffer, which cause
conversion error on near MAXPATHLEN long pathes.
Increase g_Ctoc() conversion buffers to MB_LEN_MAX - 1.
as pattern meta chars.
2) GLOB_ERR and gl_errfunc are supposed to work only for real directories
per POSIX, so don't act on missing or plain files, for ENOENT or ENOTDIR
(as TODO in the code suggested).
3) Remove the hack in the manpage describing how to skip ENOENT and ENOTDIR
in gl_errfunc, it is unneeded now.
4) Set errno to ENAMETOOLONG if g_Ctoc() expansion fails in g_opendir(),
as in other places in the code which are wrappers around system functions.
open/read errors and with GLOB_ERR and gl_errfunc processing), so we can't
blindly return it on any MAXPATHLEN overflow. Even our manpage disagrees
with such GLOB_ABORTED usage. Use GLOB_NOSPACE for that now with errno is
set to 0 as for limits.
2) Return GLOB_NOSPACE when valid ~ expansion can't happens due to
MAXPATHLEN overflow too.
3) POSIX (and our manpage) says, if GLOB_ERR is set, GLOB_ABORTED should
be returned immediatelly, without using gl_errfunc. Implement it now.
them down. This hack still remains:
* 2. Illegal byte sequences in filenames are handled by treating them as
* single-byte characters with a values of such bytes of the sequence
* cast to wchar_t.
2) Reword the comment in the hack above to reflect implementation.
3) Protect signed wchar_t from sign extension when a signed char is assigned
to it in the hack above.
3) Corresponding backward hack in g_Ctoc() was not implemented, so all
pathes with illegal byte sequences are skipped as result, implement it now.
4) globtilde() forget to convert expanded user home dir from multibyte to
wchar.
5) Protect globtilde() from long expansion truncation.
6) Results was not sorted according to collate as POSIX requires.
locale (which cause core dump) by removing whole 'table' argument
by which it passed.
2) Restore __collate_range_cmp() in __sccl().
3) Collating [a-z] range in regcomp() only for single bytes locales
(we can't do it now for other ones). In previous state only first 256
wchars are considered and all others are just silently dropped from the
range.
Instead of changing whole course to another POSIX-permitted way
for consistency and uniformity I decide to completely ignore missing
regex fucntionality and concentrace on fixing bugs in what we have now,
too many small obstacles instead, counting ports.
Only first 256 wide chars are considered currently, all other are just
dropped from the range. Proper implementation require reverse tables
database lookup, since objects are really big as max UTF-8 (1114112
code points), so just the same scanning as it was for 256 chars will
slow things down.
POSIX does not require collation for [a-z] type ranges and does not
prohibit it for non-POSIX locales. POSIX require collation for ranges
only for POSIX (or C) locale which is equal to ASCII and binary for
other chars, so we already have it.
No other *BSD implements collation for [a-z] type ranges.
Restore ABI compatibility with unused now __collate_range_cmp() which
is visible from outside (will be removed later).
Setting time by seconds or microseconds may cause unexpected effects
especially if sysctl vfs.timestamp_precision=3 (not default).
Calling the obsolete functions with NULL timestamps is acceptable.
This support appears to have been documented in nsswitch.conf(5) for some
time. The implementation adds two NSS netgroup providers to libc. The
default, compat, provides the behaviour documented in netgroup(5), so this
change does not make any user-visible behaviour changes. A files provider
is also implemented.
innetgr(3) is implemented as an optional NSS method so that providers such
as NIS which are able to implement efficient reverse lookup can do so.
A fallback implementation is used otherwise. getnetgrent_r(3) is added for
convenience and to provide compatibility with glibc and Solaris.
With a small patch to net/nss_ldap, it's possible to specify an ldap
netgroup provider, allowing one to query nisNetgroupTriple entries.
Sponsored by: EMC / Isilon Storage Division
Handle an empty result from yp_match() by returning NULL, which is
consistent with the handling of an empty netgroup in /etc/netgroup.
setnetgrent(3) has no return value, so there is no particular need to
distinguish this case from an error.
PR: 26486
MFC after: 2 weeks
This adds stravis() and some new encoding flags VIS_SHELL, VIS_META,
and VIS_NOLOCALE.
Assorted cleanups and fixes includeing a manpage typo[0].
PR: 210013 [0]
Submitted by: pi [0]
If malloc() fails to allocate linep, then free olinep (if it exists)
before returning to avoid a memory leak.
Reported by: Coverity
CID: 1016716
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D6755
When collation support was brought in, the second and third
arguments in __collate_range_cmp() were changed from int to
wchar_t, breaking the ABI. Change them to a "char" type which
makes more sense and keeps the ABI compatible.
Also introduce __wcollate_range_cmp() which does work with wide
characters. This function is used only internally in libc so
we don't export it. Use the new function in glob(3), fnmatch(3),
and regexec(3).
PR: 179721
Suggested by: ache. jilles
MFC after: 3 weeks (perhaps partial only)
POSIX 2008 added the psignal() function which has already been part of
the BSDs for a long time. The only difference is, the POSIX version uses
an 'int' for the signal number, unlike our version which uses an
'unsigned int'. Fix up the function to use an 'int'. This should not
affect the ABI.
intention of the POSIX IEEE Std 1003.1TM-2008/Cor 1-2013.
A robust mutex is guaranteed to be cleared by the system upon either
thread or process owner termination while the mutex is held. The next
mutex locker is then notified about inconsistent mutex state and can
execute (or abandon) corrective actions.
The patch mostly consists of small changes here and there, adding
neccessary checks for the inconsistent and abandoned conditions into
existing paths. Additionally, the thread exit handler was extended to
iterate over the userspace-maintained list of owned robust mutexes,
unlocking and marking as terminated each of them.
The list of owned robust mutexes cannot be maintained atomically
synchronous with the mutex lock state (it is possible in kernel, but
is too expensive). Instead, for the duration of lock or unlock
operation, the current mutex is remembered in a special slot that is
also checked by the kernel at thread termination.
Kernel must be aware about the per-thread location of the heads of
robust mutex lists and the current active mutex slot. When a thread
touches a robust mutex for the first time, a new umtx op syscall is
issued which informs about location of lists heads.
The umtx sleep queues for PP and PI mutexes are split between
non-robust and robust.
Somewhat unrelated changes in the patch:
1. Style.
2. The fix for proper tdfind() call use in umtxq_sleep_pi() for shared
pi mutexes.
3. Removal of the userspace struct pthread_mutex m_owner field.
4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls
the lifetime of the shared mutex associated with a vnode' page.
Reviewed by: jilles (previous version, supposedly the objection was fixed)
Discussed with: brooks, Martin Simmons <martin@lispworks.com> (some aspects)
Tested by: pho
Sponsored by: The FreeBSD Foundation
According to style(9):
> normally, include <sys/types.h> OR <sys/param.h>, but not both.
(<sys/param.h> already includes <sys/types.h> when LOCORE is not defined).
We have an nitems() macro in the <sys/param.h> header that is
convenient to re-use as it makes things easier to read.
Given that it is available already without adding additional
headers and other parts of libc already use it, extend a bit
more its use.
In r103767 the kern.ps_strings sysctl was added as the preferred way to
locate the ps_strings struct and is available in any FreeBSD release
supported within the last decade.
Reviewed by: kib
While here also cleanup some surrounding code; particularly
drop some malloc() casts.
Found with devel/coccinelle.
Reviewed by: bde (previous version - all new bugs are mine)
do not participate in the global symbols namespace, but rtld locks are
still replaced and functions are interposed. In particular,
__pthread_map_stacks_exec is resolved to the libc version. If a
library is loaded later, which requires adjustment of the stack
protection mode, rtld calls into libc __pthread_map_stacks_exec due to
the symbols scope. The libc version might recurse into binder and
recursively acquire rtld bind lock, causing the hang.
Make libc __pthread_map_stacks_exec() interposed, which synchronizes
rtld locks and version of the stack exec hook when libthr loaded,
regardless of the symbol scope control or symbol resolution order.
The __pthread_map_stacks_exec() symbol is removed from the private
version in libthr since libc symbol now operates correctly in presence
of libthr.
Reported and tested by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
wraps sendmsg(2) and recvmsg(2) into batch send and receive operation.
The goal of this implementation is only to provide API compatibility
with Linux.
The cancellation behaviour of the functions is not quite right, but
due to relative rare use of cancellation it is considered acceptable
comparing with the complexity of the correct implementation. If
functions are reimplemented as syscalls, the fix would come almost
trivial. The direct use of the syscall trampolines instead of libc
wrappers for sendmsg(2) and recvmsg(2) is to avoid data loss on
cancellation.
Submitted by: Boris Astardzhiev <boris.astardzhiev@gmail.com>
Discussed with: jilles (cancellation behaviour)
MFC after: 1 month
by application closing its stdin (i.e. STDIN_FILENO) prior to
calling readpassphrase WITHOUT setting RPP_STDIN. What happens
then is that the readpassphrase would open /dev/tty, and since
file descriptors are reused, the call would return first unused
fd, which is 0 which is also STDIN_FILENO. Then due to the usage
of "input != STDIN_FILENO" in the code to do its logic, that
would result in noecho flags not set on that file descriptor,
which was original issue I've been trying to fix.
In addition to that, the readpassphrase() would leak file
descriptor on its way out, so fix that one as well.
This problem can be tested with:
$ ssh-add - < /tmp/myprivate.key
The password will not be hidden as it should and ktrace will
show:
53326 ssh-add CALL open(0x80142443c,0x100002<O_RDWR|O_CLOEXEC>,<unused>0x165f030)
53326 ssh-add NAMI "/dev/tty"
53326 ssh-add RET open 0
53326 ssh-add CALL sigprocmask(SIG_SETMASK,0x802eb1324,0x7fffffffd5e0)
53326 ssh-add RET sigprocmask 0
53326 ssh-add CALL sigaction(SIGALRM,0x7fffffffd630,0x7fffffffd610)
Instead of:
57690 ssh-add CALL open(0x80142443c,0x100002<O_RDWR|O_CLOEXEC>,<unused>0x165f030)
57690 ssh-add NAMI "/dev/tty"
57690 ssh-add RET open 4
57690 ssh-add CALL ioctl(0x4,TIOCGETA,0x7fffffffd860)
57690 ssh-add RET ioctl 0
57690 ssh-add CALL ioctl(0x4,TIOCSETAF,0x7fffffffd680)
57690 ssh-add RET ioctl 0
57690 ssh-add CALL sigprocmask(SIG_SETMASK,0x802eb1324,0x7fffffffd620)
57690 ssh-add RET sigprocmask 0
57690 ssh-add CALL sigaction(SIGALRM,0x7fffffffd670,0x7fffffffd650)
For the case when the key is read from the file.
Technically this can also be workaround'ed at the application side
by not closing the STDIN_FILENO in the first place, but readpassphrase(3)
doesn't need to make any assumptions about that. Plus the file descriptor
leak confirms that this is an oversight, rather than a deliberate behaviour.
MFC after: 1 week
This bug could be reproduced easily by calling sem_open() with O_CREAT |
O_EXCL on a semaphore that is already open in the process. The struct
sem_nameinfo would be freed while still in sem_list and later calls to
sem_open() or sem_close() could access freed memory.
PR: 206396
MFC after: 5 days
Set _PATH_DEFPATH to
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin. This is the
path in the default class in the default /etc/login.conf,
excluding ~/bin which would not be expanded properly in a string
constant.
For normal logins, _PATH_DEFPATH is overridden by /etc/login.conf,
~/.login_conf or shell startup files. _PATH_DEFPATH is still used as a
default by execlp(), execvp(), posix_spawnp() and sh if PATH is not set, and
by cron. Especially the latter is a common trap (most recently in PR
204813).
PR: 204813
Reviewed by: secteam (delphij), alfred
i386 is the only current FreeBSD architecture that ever used a.out
format.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D4687
Tracking these leads to situations where meta mode will consider the
file to be out of date if /bin/sh or /bin/ln are newer than the source
file. There's no reason for meta mode to do this as make is already
handling the rebuild dependency fine.
Sponsored by: EMC / Isilon Storage Division
Move fdopen() up near other resource allocation like malloc(); do proper
deallocation on failure later on in the function.
Submitted by: Ramachandra Topannavar <rtopannavar@panasas.com>
Reviewed by: jilles
Approved by: jhb (mentor)
MFC after: 2 weeks
Sponsored by: Panasas, Inc.
Differential Revision: https://reviews.freebsd.org/D4126
M lib/libc/gen/popen.c
MK_NIS == no by converting `i` back to an int, and instead cast the loop
comparison to `int`
The loop comparison is iterating the len(ns_dtab)-1, because
the last element is the sentinel tuple { NULL, NULL, NULL, }, so when
both HESOID and NIS are off, len(ns_dtab)-1 == 1 - 1 == 0, and the loop
is skipped because the expression is tautologically false
While here, convert `(sizeof(x) / sizeof(x[0]))` to `nitems(x)`
Tested with: clang 3.7.0, gcc 4.2.1, and gcc 4.9.4 [*] with MK_NIS={no,yes}
and by running bash -lc 'id -u && id -g && id'
* gcc 4.9.4 needs another patch in order for the compile to succeed
with -Werror with lib/libc/gen/getgrent.c
Reported by: jhibbits
Shell syntax is too complicated to detect command substitution and unquoted
operators reliably without implementing much of sh's parser. Therefore, have
sh do this detection.
While changing sh's support anyway, also read input from a pipe instead of
arguments to avoid {ARG_MAX} limits and improve privacy, and output count
and length using 16 instead of 8 digits.
The basic concept is:
execl("/bin/sh", "sh", "-c", "freebsd_wordexp ${1:+\"$1\"} -f "$2",
"", flags & WRDE_NOCMD ? "-p" : "", <pipe with words>);
The WRDE_BADCHAR error is still implemented in libc. POSIX requires us to
fail strings containing unquoted braces with code WRDE_BADCHAR. Since this
is normally not a syntax error in sh, there is still a need for checking
code in libc, we_check().
The new we_check() is an optimistic check that all the characters
<newline> | & ; < > ( ) { }
are quoted. To avoid duplicating too much sh logic, such characters are
permitted when quoting characters are seen, even if the quoting characters
may themselves be quoted. This code reports all WRDE_BADCHAR errors; bad
characters that get past it and are a syntax error in sh return WRDE_SYNTAX.
Although many implementations of WRDE_NOCMD erroneously allow some command
substitutions (and ours even documented this), there appears to be code that
relies on its security (codesearch.debian.net shows quite a few uses).
Passing untrusted data to wordexp() still exposes a denial of service
possibility and a fairly large attack surface.
Reviewed by: wblock (man page only)
MFC after: 2 weeks
Relnotes: yes
Security: fixes command execution with wordexp(untrusted, WRDE_NOCMD)
The old code was exponential in the number of asterisks in the pattern.
However, once a match has been found upto the next asterisk, the previous
asterisks are no longer relevant.
comment above, POSIX_SPAWN_SETSIGMASK and POSIX_SPAWN_SETSIGDEF
handlers used libthr interposed functions instead of syscalls.
Noted by: jilles
Sponsored by: The FreeBSD Foundation
MFC after: 6 days
are aliases for the syscall stubs and are plt-interposed, to the
libc-private aliases of internally interposed sigprocmask() etc.
Since e.g. _sigaction is not interposed by libthr, calling signal()
removes thr_sighandler() from the handler slot etc. The result was
breaking signal semantic and rtld locking.
The added __libc_sigprocmask and other symbols are hidden, they are
not exported and cannot be called through PLT. The setjmp/longjmp
functions for x86 were changed to use direct calls, and since
PIC_PROLOGUE only needed for functional PLT indirection on i386, it is
removed as well.
The PowerPC bug of calling the syscall directly in the setjmp/longjmp
implementation is kept as is.
Reported by: Pete French <petefrench@ingresso.co.uk>
Tested by: Michiel Boland <boland37@xs4all.nl>
Reviewed by: jilles (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Distinguish between WRDE_BADVAL and WRDE_SYNTAX based on when the error
occurred (parsing or execution), not based on whether WRDE_UNDEF was passed.
Also, return WRDE_NOSPACE for a few more unexpected results from sh.
The functionality of the wordexp builtin is easily replaced using normal
shell code, although performance is slightly worse.
This does not mean that wordexp() will remain shell-independent -- a fully
reliable implementation of WRDE_NOCMD is really only possible using
extensions to the shell, or by adding much of the shell's code to libc.
the 'user' sysctl tree, which have all been coming back 0 or empty
since r240176.
Differential Revision: https://reviews.freebsd.org/D2945
Reviewed by: sbruno
Approved by: jmallett (mentor)
MFC after: 3 days
as seek to teh last location saved will still work. This is needed for Samba
to be able to correctly handle delete requests from windows. This does not
completely fix seekdir when deletes are present but fixes the worst of the
problems. The real solution must involve some changes to the API for eh VFS
and getdirentries(2).
Obtained from: Panzura inc
MFC after: 1 week
kernel, but keep explanation of the old ps_strings structure to make
it clear what sanity check tries to accomplish.
Noted by: Oliver Pinter <oliver.pinter@hardenedbsd.org>
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
waitid() function is required to be cancellable by the standard. The
wait6() and ppoll() follow the other syscalls in their groups.
Reviewed by: jhb, jilles (previous versions)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
dependent functions have been implemented, but this is enough for world.
Differential Revision: https://reviews.freebsd.org/D2132
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
When following symlinks, fts returned FTS_SLNONE when fstatat(flag=0)
failed, but a subsequent fstatat(flag=AT_SYMLINK_NOFOLLOW) succeeded. This
incorrectly triggered if a filename existed to be read from the directory,
was deleted before the fstatat(flag=0) and created again after the
fstatat(flag=0).
Fix this by only returning FTS_SLNONE if the result from
fstatat(flag=AT_SYMLINK_NOFOLLOW) is actually a symlink. If it is not a
symlink, treat it as if fstatat(flag=0) succeeded.
PR: 196724
Reported and tested by: pho
MFC after: 1 week
Per Austin group issue #884, sh should not import IFS from the environment
but always set it to $' \t\n'. For wordexp(), however, it is documented and
useful for it to use IFS from the environment.
Since sh currently imports IFS from the environment, this change has no
functional effect.
MFC after: 1 week
Commit r279154 changed the API and ABI significantly, and {NZERO} is still
wrong.
Also, preserve errno on success instead of setting it to 0.
PR: 189821
Reported by: bde
Relnotes: yes
Our man page already documented this partially but now
we have some consistent behavior.
PR: 136669
Obtained from: NetBSD (CVS rev. 1.31, 1.33)
Relnotes: yes
MFC after: 3 weeks
rlim_t is at least as large as long, so we don't need the
extra variable to keep the intermediate step. We don't
need the volatile either.
The code was tested on i386 and amd64.
Suggested by: bde
X-MFC with: r278803
As a followup to r278363, there is one more case where
stayopen can be accessed uninitialized, but even after
swapping arguments, access is possible in some other
cases so prevent it completely by initializing stayopen.
CID: 1018729
CID: 1018732
The existing implementation had a broken comparison that could
overflow and return confusing values. Replace this with a check
that avoids the overflow before it happens.
Consistently return a maximum value also on the case of negative
arguments since negative is considered an overflow and means
infinity for our current setrlimit().
New revamped version is credited to Bruce Evans.
CID: 1199295
MFC after: 1 week
particular, stdio locking was affected.
Reported and tested by: "Matthew D. Fuller" <fullermd@over-yonder.net>
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
The existing implementation had a broken comparison that could overflow.
Replace this with a check that avoids the overflow before it happens.
Consistently return a maximum value also on the case of negative
arguments since negative is considered an overflow and means
infinity for our current setrlimit().
Discussed with: bde (rather extensively)
CID: 1199295
MFC after: 1 week
In a couple of cases a variable "stayopen" can be checked
unitialized. This is of no danger as the complementary
condition is false but prevent the access by switching
the checks.
CID: 1018729
CID: 1018732
This was a local addition to the original change from NetBSD.
Being this libc there is some chance for it to interfere with
user's cget*() functions usage. The memory leak was finely
plugged by r278300.
Pointed out by: ache
Only i386 and amd64 provide a non-trivial __getcontextx(). Use a common
trivial implementation in gen/ for other architectures, rather than
copying the file to each MD subdirectory.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1472
cancellation-handling code in the libthr. Translate some syscalls
into their more generic counterpart, and remove translated syscalls
from the table.
List of the affected syscalls:
creat, open -> openat
raise -> thr_kill
sleep, usleep -> nanosleep
pause -> sigsuspend
wait, wait3, waitpid -> wait4
Suggested and reviewed by: jilles (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
(or loading a dso linked to libthr.so into process which was not
linked against threading library).
- Remove libthr interposers of the libc functions, including
__error(). Instead, functions calls are indirected through the
interposing table, similar to how pthread stubs in libc are already
done. Libc by default points either to syscall trampolines or to
existing libc implementations. On libthr load, libthr rewrites the
pointers to the cancellable implementations already in libthr. The
interposition table is separate from pthreads stubs indirection
table to not pull pthreads stubs into static binaries.
- Postpone the malloc(3) internal mutexes initialization until libthr
is loaded. This avoids recursion between calloc(3) and static
pthread_mutex_t initialization.
- Reinstall signal handlers with wrapper on libthr load. The
_rtld_is_dlopened(3) is used to avoid useless calls to sigaction(2)
when libthr is statically referenced from the main binary.
In the process, fix openat(2), swapcontext(2) and setcontext(2)
interposing. The libc symbols were exported at different versions
than libthr interposers. Export both libc and libthr versions from
libc now, with default set to the higher version from libthr.
Remove unused and disconnected swapcontext(3) userspace implementation
from libc/gen.
No objections from: deischen
Tested by: pho, antoine (exp-run) (previous versions)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
In r228193 the test of CONNPRIV have been moved to before the _usleep
and send in vsyslog(). When syslogd restarts, this would prevent the
message being logged after the disconnect/connect dance for
scenario #1.
PR: 194751
Submitted by: Peter Creath <pjcreath+freebsd gmail com>
Reviewed By: glebius
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D1227
It is automatically set when -fPIC is passed to the compiler.
Reviewed by: dim, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1179
in a separate word from the _count. This does not permit both items to
be updated atomically in a portable manner. As a result, sem_post()
must always perform a system call to safely clear _has_waiters.
This change removes the _has_waiters field and instead uses the high bit
of _count as the _has_waiters flag. A new umtx object type (_usem2) and
two new umtx operations are added (SEM_WAIT2 and SEM_WAKE2) to implement
these semantics. The older operations are still supported under the
COMPAT_FREEBSD9/10 options. The POSIX semaphore API in libc has
been updated to use the new implementation. Note that the new
implementation is not compatible with the previous implementation.
However, this only affects static binaries (which cannot be helped by
symbol versioning). Binaries using a dynamic libc will continue to work
fine. SEM_MAGIC has been bumped so that mismatched binaries will error
rather than corrupting a shared semaphore. In addition, a padding field
has been added to sem_t so that it remains the same size.
Differential Revision: https://reviews.freebsd.org/D961
Reported by: adrian
Reviewed by: kib, jilles (earlier version)
Sponsored by: Norse
previous seek location was 0. Without this, readdir() would see
dd_loc of zero and call getdirentries() which would start reading
entries at the current seek location of the directory ignoring the
first batch of entries. Also, rewinddir() should always seek so that
it reads the directory from the beginning to get updated entries.
PR: 192935
Reported by: iron@mail.ua
MFC after: 3 days
requires the return value of telldir() to equal the value passed to
seekdir(). The current seekdir code with SINGLEUSE enabled breaks
this case as each call to telldir() allocates a new cookie. Instead,
remove the SINGLEUSE code and change telldir() to look for an existing
cookie for the directory's current location rather than always creating
a new cookie.
CR: https://phabric.freebsd.org/D490
PR: 121656
Reviewed by: jilles
MFC after: 1 week
after an intervening call to rewinddir() is undefined, so reclaim any
pending telldir() cookies in the directory when rewinddir() is called.
CR: D459
Reviewed by: jilles
MFC after: 1 week
arc4random.c
- CVS rev. 1.22
Change arc4random_uniform() to calculate ``2**32 % upper_bound'' as
``-upper_bound % upper_bound''. Simplifies the code and makes it the
same on both ILP32 and LP64 architectures, and also slightly faster on
LP64 architectures by using a 32-bit remainder instead of a 64-bit
remainder.
- CVS rev. 1.23
Spacing
readpassphrase.c
-CVS rev. v 1.24
most obvious unsigned char casts for ctype
Obtained from: OpenBSD
MFC after: 5 days
- In the unionfs case, opendir() and fdopendir() read the directory's full
contents and cache it. This cache is not refreshed when rewinddir() is
called, so rewinddir() will not notice updates to a directory. Fix this
by splitting the code to fetch a directory's contents out of
__opendir_common() into a new _filldir() function and call this from
rewinddir() when operating on a unionfs directory.
- If rewinddir() is called on a directory opened with fdopendir() before
any directory entries are fetched, rewinddir() will not adjust the seek
location of the backing file descriptor. If the file descriptor passed
to fdopendir() had a non-zero offset, the rewinddir() will not rewind to
the beginning. Fix this by always seeking back to 0 in rewinddir().
This means the dd_rewind hack can also be removed.
While here, add missing locking to rewinddir().
CR: https://phabric.freebsd.org/D312
Reviewed by: jilles
MFC after: 1 week
This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation
This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h
Discussed at: BSDcan
actual file storing the semaphore object is different from the file
created on the first open. Store the file st_dev and st_ino members
of the struct stat in the semaphore structure on open, and compare
them with the attributes of the opened file to detect unlink and
re-creation.
This fixes an issue of sem_unlink(3) failing to flush the named entry
in the semaphore list for the current or remote process, making
sem_unlink(3) not correctly operating if the unlinked semaphore is
still opened.
Reported by: Joris Giovannangeli <joris@giovannangeli.fr>
PR: standards/189353
Reviewed by: jilles (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
pointer for the login name (result). Make sure to handle that
case properly. Improve robustness by checking namelen and then
nul-terminating the provided buffer to simplify subsequent logic.
Obtained from: Juniper Networks, Inc.
MFC after: 1 week
Make fts_open(3) treat an empty pathname like any other pathname that cannot
be lstatted because of [ENOENT].
It is rather confusing if rm -rf file1 "" file2 does not remove file1 and
file2.
PR: bin/187264
MFC after: 2 weeks
if not already defined. This allows building libc from outside of
lib/libc using a reach-over makefile.
A typical use-case is to build a standard ILP32 version and a COMPAT32
version in a single iteration by building the COMPAT32 version using a
reach-over makefile.
Obtained from: Juniper Networks, Inc.
If rare conditions such as concurrent conflicting manipulation of the
filesystem occur, fts_read() frees the current FTSENT without adjusting
the pointers in the FTS accordingly. A later fts_close() then frees the
same FTSENT again.
Reported by: pho
Tested by: pho
MFC after: 1 week
device is an active kernel console and "off" otherwise. This is designed to
allow serial-booting x86 systems to provide a login prompt on the serial line
by default without providing one on all systems by default.
Comments and suggestions by: grehan, dteske, jilles
MFC after: 1 month
As a result, the kernel needs to process shorter pathnames if fts is not
changing directories (if fts follows symlinks (-L option to utilities), fts
cannot open "." or FTS_NOCHDIR was specified).
Side effect: If pathnames exceed PATH_MAX, [ENAMETOOLONG] is not hit at the
stat stage but later (opendir or application fts_accpath) or not at all.
Even though not all race conditions can be fixed if the 'e' option is not
used, still fix some race conditions using pipe2():
* Prevent both ends of the pipe from leaking to a concurrent popen().
* Prevent the child process's end of the pipe from leaking to any concurrent
fork and exec.
This change also simplifies the code.
This ensures strerror() and friends continue to work correctly even if a
(non-PIE) executable linked against an older libc imports sys_errlist (which
causes sys_errlist to refer to the executable's copy with a size fixed when
that executable was linked).
The executable's use of sys_errlist remains broken because it uses the
current value of sys_nerr and may access past the bounds of the array.
Different from the message "Using sys_errlist from executables is not
ABI-stable" on freebsd-arch, this change does not affect the static library.
There seems no reason to prevent overriding the error messages in the static
library.
and CIFS file attributes as BSD stat(2) flags.
This work is intended to be compatible with ZFS, the Solaris CIFS
server's interaction with ZFS, somewhat compatible with MacOS X,
and of course compatible with Windows.
The Windows attributes that are implemented were chosen based on
the attributes that ZFS already supports.
The summary of the flags is as follows:
UF_SYSTEM: Command line name: "system" or "usystem"
ZFS name: XAT_SYSTEM, ZFS_SYSTEM
Windows: FILE_ATTRIBUTE_SYSTEM
This flag means that the file is used by the
operating system. FreeBSD does not enforce any
special handling when this flag is set.
UF_SPARSE: Command line name: "sparse" or "usparse"
ZFS name: XAT_SPARSE, ZFS_SPARSE
Windows: FILE_ATTRIBUTE_SPARSE_FILE
This flag means that the file is sparse. Although
ZFS may modify this in some situations, there is
not generally any special handling for this flag.
UF_OFFLINE: Command line name: "offline" or "uoffline"
ZFS name: XAT_OFFLINE, ZFS_OFFLINE
Windows: FILE_ATTRIBUTE_OFFLINE
This flag means that the file has been moved to
offline storage. FreeBSD does not have any special
handling for this flag.
UF_REPARSE: Command line name: "reparse" or "ureparse"
ZFS name: XAT_REPARSE, ZFS_REPARSE
Windows: FILE_ATTRIBUTE_REPARSE_POINT
This flag means that the file is a Windows reparse
point. ZFS has special handling code for reparse
points, but we don't currently have the other
supporting infrastructure for them.
UF_HIDDEN: Command line name: "hidden" or "uhidden"
ZFS name: XAT_HIDDEN, ZFS_HIDDEN
Windows: FILE_ATTRIBUTE_HIDDEN
This flag means that the file may be excluded from
a directory listing if the application honors it.
FreeBSD has no special handling for this flag.
The name and bit definition for UF_HIDDEN are
identical to the definition in MacOS X.
UF_READONLY: Command line name: "urdonly", "rdonly", "readonly"
ZFS name: XAT_READONLY, ZFS_READONLY
Windows: FILE_ATTRIBUTE_READONLY
This flag means that the file may not written or
appended, but its attributes may be changed.
ZFS currently enforces this flag, but Illumos
developers have discussed disabling enforcement.
The behavior of this flag is different than MacOS X.
MacOS X uses UF_IMMUTABLE to represent the DOS
readonly permission, but that flag has a stronger
meaning than the semantics of DOS readonly permissions.
UF_ARCHIVE: Command line name: "uarch", "uarchive"
ZFS_NAME: XAT_ARCHIVE, ZFS_ARCHIVE
Windows name: FILE_ATTRIBUTE_ARCHIVE
The UF_ARCHIVED flag means that the file has changed and
needs to be archived. The meaning is same as
the Windows FILE_ATTRIBUTE_ARCHIVE attribute, and
the ZFS XAT_ARCHIVE and ZFS_ARCHIVE attribute.
msdosfs and ZFS have special handling for this flag.
i.e. they will set it when the file changes.
sys/param.h: Bump __FreeBSD_version to 1000047 for the
addition of new stat(2) flags.
chflags.1: Document the new command line flag names
(e.g. "system", "hidden") available to the
user.
ls.1: Reference chflags(1) for a list of file flags
and their meanings.
strtofflags.c: Implement the mapping between the new
command line flag names and new stat(2)
flags.
chflags.2: Document all of the new stat(2) flags, and
explain the intended behavior in a little
more detail. Explain how they map to
Windows file attributes.
Different filesystems behave differently
with respect to flags, so warn the
application developer to take care when
using them.
zfs_vnops.c: Add support for getting and setting the
UF_ARCHIVE, UF_READONLY, UF_SYSTEM, UF_HIDDEN,
UF_REPARSE, UF_OFFLINE, and UF_SPARSE flags.
All of these flags are implemented using
attributes that ZFS already supports, so
the on-disk format has not changed.
ZFS currently doesn't allow setting the
UF_REPARSE flag, and we don't really have
the other infrastructure to support reparse
points.
msdosfs_denode.c,
msdosfs_vnops.c: Add support for getting and setting
UF_HIDDEN, UF_SYSTEM and UF_READONLY
in MSDOSFS.
It supported SF_ARCHIVED, but this has been
changed to be UF_ARCHIVE, which has the same
semantics as the DOS archive attribute instead
of inverse semantics like SF_ARCHIVED.
After discussion with Bruce Evans, change
several things in the msdosfs behavior:
Use UF_READONLY to indicate whether a file
is writeable instead of file permissions, but
don't actually enforce it.
Refuse to change attributes on the root
directory, because it is special in FAT
filesystems, but allow most other attribute
changes on directories.
Don't set the archive attribute on a directory
when its modification time is updated.
Windows and DOS don't set the archive attribute
in that scenario, so we are now bug-for-bug
compatible.
smbfs_node.c,
smbfs_vnops.c: Add support for UF_HIDDEN, UF_SYSTEM,
UF_READONLY and UF_ARCHIVE in SMBFS.
This is similar to changes that Apple has
made in their version of SMBFS (as of
smb-583.8, posted on opensource.apple.com),
but not quite the same.
We map SMB_FA_READONLY to UF_READONLY,
because UF_READONLY is intended to match
the semantics of the DOS readonly flag.
The MacOS X code maps both UF_IMMUTABLE
and SF_IMMUTABLE to SMB_FA_READONLY, but
the immutable flags have stronger meaning
than the DOS readonly bit.
stat.h: Add definitions for UF_SYSTEM, UF_SPARSE,
UF_OFFLINE, UF_REPARSE, UF_ARCHIVE, UF_READONLY
and UF_HIDDEN.
The definition of UF_HIDDEN is the same as
the MacOS X definition.
Add commented-out definitions of
UF_COMPRESSED and UF_TRACKED. They are
defined in MacOS X (as of 10.8.2), but we
do not implement them (yet).
ufs_vnops.c: Add support for getting and setting
UF_ARCHIVE, UF_HIDDEN, UF_OFFLINE, UF_READONLY,
UF_REPARSE, UF_SPARSE, and UF_SYSTEM in UFS.
Alphabetize the flags that are supported.
These new flags are only stored, UFS does
not take any action if the flag is set.
Sponsored by: Spectra Logic
Reviewed by: bde (earlier version)
function, but returns directory file descriptor instead of closing it.
Submitted by: Mariusz Zaborski <oshogbo@FreeBSD.org>
Sponsored by: Google Summer of Code 2013
The variable _logname_valid is not exported via the version script;
therefore, change C and i386/amd64 assembler code to remove indirection
(which allowed interposition). This makes the code slightly smaller and
faster.
Also, remove #define PIC_GOT from i386/amd64 in !PIC mode. Without PIC,
there is no place containing the address of each variable, so there is no
possible definition for PIC_GOT.
I removed functionality not proposed for POSIX in Austin group issue #411.
A man page (my own) and test cases will follow in later commits.
PR: 176233
Submitted by: Jukka Ukkonen
any character including '\0', but our version replace escaped '\0'
with '\\'.
I.e. fnmatch("\\", "\\", 0) should not match while fnmatch("\\", "", 0)
should (Linux and NetBSD does the same). Was vice versa.
PR: 181129
MFC after: 1 week
check_deferred_signal() returns twice, since handle_signal() emulates
the return from the normal signal handler by sigreturn(2)ing the
passed context. Second return is performed on the destroyed stack
frame, because __fillcontextx() has already returned. This causes
undefined and bad behaviour, usually the victim thread gets SIGSEGV.
Avoid nested frame and the need to return from it by doing direct call
to getcontext() in the check_deferred_signal() and using a new private
libc helper __fillcontextx2() to complement the context with the
extended CPU state if the deferred signal is still present.
The __fillcontextx() is now unused, but is kept to allow older
libthr.so to be used with the new libc.
Mark __fillcontextx() as returning twice [1].
Reported by: pgj
Pointy hat to: kib
Discussed with: dim
Tested by: pgj, dim
Suggested by: jilles [1]
MFC after: 1 week
If 'e' is used, the kernel must support the recently added pipe2() system
call.
The use of pipe2() with O_CLOEXEC also fixes race conditions between
concurrent popen() calls from different threads, even if the close-on-exec
flag on the fd of the returned FILE is later cleared (because popen() closes
all file descriptors from earlier popen() calls in the child process).
Therefore, this approach should be used in all cases when pipe2() can be
assumed present.
The old version of popen() rejects "re" and "we" but treats "r+e" like "r+".
The ability to clear a file descriptor's close-on-exec flag via
posix_spawn_file_actions_adddup2() is in fact proposed in Austin Group issue
#411.
MFC after: 1 week
As per POSIX.1-2008, posix_spawn_file_actions_add* return [EBADF] if a file
descriptor is negative, not [EINVAL]. The bug was only in the manual page;
the code is correct.
MFC after: 1 week
The functions utx_active_add(), utx_active_remove(), utx_lastlogin_add() and
utx_log_add() set errno to 0 if they are successful. This not only violates
POSIX if pututxline() is successful, but may also overwrite a valid error
with 0 if, for example, utx_lastlogin_add() fails while utx_log_add()
succeeds.
Reviewed by: ed
signal.
- Fix the old ksem implementation for POSIX semaphores to not restart
sem_wait() or sem_timedwait() if interrupted by a signal.
MFC after: 1 week
Words in shell script are separated by spaces or tabs independent of the
value of IFS. The value of IFS is only relevant for the result of
substitutions. Therefore, there should be a space between 'wordexp' and the
words to be expanded, not an IFS character.
Paranoia might dictate that the shell ignore IFS from the environment (even
though our sh currently uses it), so do not depend on it in the new test
case.
multibyte support[0] and the new functions strenvisx and strsenvisx.
Add MLINKS for vis(3) functions add by this and the initial import from
NetBSD[1].
PR: bin/166364, bin/175418
Submitted by: "J.R. Oldroyd" <fbsd@opal.com>[0]
stefanf[1]
Obtained from: NetBSD
MFC after: 2 weeks
system call, which has a nice property - it never fails, so it is a bit
easier to use. If there is no support for capability mode in the kernel
the function will return false (not in a sandbox). If the kernel is compiled
with the support for capability mode, the function will return true or false
depending if the calling process is in the capability mode sandbox or not
respectively.
Sponsored by: The FreeBSD Foundation
* Reopen the directory using openat(fd, ".", ...) instead of opening the
pathname again. This fixes a race condition where the meaning of the
pathname changes and allows a reopen with fdopendir().
* Always reopen the directory for union stacks, not only when DTF_REWIND
is passed. Applications should be able to fchdir(dirfd(dir)) and
*at(dirfd(dir), ...). DTF_REWIND now does nothing.
