According to the getpeereid(3) documentation, on failure the value -1 is
returned and the global variable errno is set to indicate the error. We
were returning the error instead.
Obtained from: Apple's Libc-1244.30.3
MFC after: 5 days
utilities is done by calling gr_addgid() for each group to be
added (usually found by traversing /etc/group) then calling the
setgroups() system call after the group set has been created.
The gr_addgid() function (helpfully?) deduplicates the addition
of group members. So, if you call it to add a group member that
already exists, it is just dropped. Because group[0] is the
effective group-ID and is over-written when a setgid program
is run, The value in group[0] is usually duplicated so that
group value is not lost when a setgid program is run.
Historically this happened because the group value indicated
in the password file also appears in /etc/group (e.g., if you
are group staff in the password file, you will also appear in
the staff line in /etc/group). But, with the addition of the
deduplication, the attempt to add group staff was lost because
it already appeared in group[0]. So, the fix is to deduplicate
starting from group[1] which allows a duplicate of the entry in
group[0], but not in later entries.
There is some confusion about the setgroups system call because in
BSD it has (always) set the entire group including the egid group
(in group[0]). However, in Linux, it skips over group[0] and starts
setting from group[1]. See this comment from linux_setgroups:
/*
* cr_groups[0] holds egid. Setting the whole set from
* the supplied set will cause egid to be changed too.
* Keep cr_groups[0] unchanged to prevent that.
*/
To make it clear what the BSD setgroups system call does, I
added the following paragraph to the setgroups(2) manual page:
The first entry of the group array (gidset[0]) is used as the effective
group-ID for the process. This entry is over-written when a setgid
program is run. To avoid losing access to the privileges of the
gidset[0] entry, it should be duplicated later in the group array.
By convention, this happens because the group value indicated in the
password file also appears in /etc/group. The group value in the
password file is placed in gidset[0] and that value then gets added a
second time when the /etc/group file is scanned to create the group set.
Reported by: Paul McMath paulm at tetrardus.net
Reviewed by: kib
MFC after: 2 weeks
The daemonfd function is equivalent to the daemon(3) function expect that
arguments are descriptors. For example dhclient(8) which is sandboxed is
unable to open /dev/null to close stdio instead it's allows to fail
daemon(3) function to close the descriptors and then do it explicit in code.
Instead of such hacks we can use now daemonfd.
This API can be also helpful to migrate system to platforms like CheriBSD.
Reviewed by: brooks@, bcr@, jilles@ (earlier version)
Differential Revision: https://reviews.freebsd.org/D13433
There are two versions of variant I of TLS
- ARM and aarch64 uses original version of variant I here TP points to
start of TCB followed by aligned TLS segment. Both TCB and TLS must
be aligned to alignment of TLS section. The TCB[0] points to DTV vector
and DTV values are real addresses (without bias).
- MIPS, PowerPC and RISC-V use modified version of variant I,
where TP points (with bias) to TLS and TCB immediately precedes TLS
without any alignment gap. Only TLS should be aligned. The TCB[0]
points to DTV vector and DTV values are biased by constant value (0x8000)
from real addresses.
Take all this in account when allocating memory for TLS structures.
MFC after: 1 month
Reviewed by: kib, mizhka
Tested by: mizhka(on mips)
Differential Revision: https://reviews.freebsd.org/D13378
Now that the POSIX working group is going to require that basename(3)
and dirname(3) are thread-safe in future revisions of the standard,
there is even less of a need to provide basename_r(3). Remove this
function to prevent people from writing code that only builds on
FreeBSD and Bionic.
Removing this function seems to break exactly one port: sbruno@'s
qemu-user-static. I will send him a pull request on GitHub in a bit.
__FreeBSD_version will not be bumped, as any value from 2017 can be used
to test for the presence of a thread-safe basename(3)/dirname(3).
PR: https://bugs.freebsd.org/224016
Currently each call to telldir() requires a malloc and adds an entry to a
linked list which must be traversed on future telldir(), seekdir(),
closedir(), and readdir() calls. Applications that call telldir() for every
directory entry incur O(n^2) behavior in readdir() and O(n) in telldir() and
closedir().
This optimization eliminates the malloc() and linked list in most cases by
packing the relevant information into a single long. On 64-bit architectures
msdosfs, NFS, tmpfs, UFS, and ZFS can all use the packed representation. On
32-bit architectures msdosfs, NFS, and UFS can use the packed
representation, but ZFS and tmpfs can only use it for about the first 128
files per directory. Memory savings is about 50 bytes per telldir(3) call.
Speedup for telldir()-heavy directory traversals is about 20-30x for one
million files per directory.
Reviewed by: kib, mav, mckusick
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13385
system calls. Man pages are missing for v2 and v5, so any entries for
those versions were inferred by new implementations of these functions
in libc.
Obtained from: http://www.tuhs.org/cgi-bin/utree.pl
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
For statically linked binaries, where all relocation are solved by static
linker, the linker expect that offset to TLS section is aligned. Additionaly,
to maintain absolute alignment, TLS TCB should by also aligned.
Obtained from: CheriBSD (initial version)
MFC after: 1 month
Reviewed by: brooks (previous version), kib
Differential Revision: https://reviews.freebsd.org/D12907
one call to sysctl(2) from jemalloc startup code. (That also requires
changes to jemalloc, but I plan to push those to upstream first.)
Reviewed by: kib
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12745
- Teach elf aux vector functions about newly added AT_HWCAP and AT_HWCAP2
vectors.
- Export _elf_aux_info() as new public libc function elf_aux_info(3)
The elf_aux_info(3) should be considered as FreeBSD counterpart of glibc
getauxval() with more robust interface.
Note:
We cannot name this new function as getauxval(), with glibc compatible
interface. Some ports autodetect its existence and then expects that all
Linux specific AT_<*> vectors are defined and implemented.
MFC after: 1 month
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D12743
On Variant I TLS architectures (aarch64, arm, mips, powerpc, and riscv)
the __libc_allocate_tls function allocates thread local storage memory
with calloc(). It then copies initialization data over the portions with
non-zero initial values. Before this change it would then pointlessly
zero the already zeroed remainder of the storage. Unfortunately the
calculation was wrong and it would zero TLS_TCB_SIZE (2*sizeof(void *))
additional bytes.
In practice, this overflow only matters if the TLS segment is sized such
that calloc() allocates a less than TLS_TCB_SIZE extra memory. Even
then, the likely result will be zeroing part of the next bucket. This
coupled with the impact being confined to Tier II platforms means there
will be no security advisory for this issue.
Reviewed by: kib, dfr
Discussed with: security-officer (delphij)
MFC after: 1 week
Found by: CHERI
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12547
Illumos and Schillix is adopting some of the locale code and our style(9)
sometimes matches the Solaris cstyle, so the changes are also useful as a
way to reduce diffs.
No functional change.
Discussed with: Joerg Schilling
MFC after: 1 week
getmntinfo(3) is designed around a relatively static or slow growing set of
current mounts. It tried to detect a race with somewhat concurrent mount
and re-call getfsstat(2) in that case, looping indefinitely. It also
allocated space for a single extra mount as slop.
In the case where the user has a large number of mounts and is adding them
at a rapid pace, it fell over.
This patch makes two functional changes:
1. Allocate even more slop. Double whatever the last getfsstat(2) returned.
2. Abort and return some known results after looping a few times
(arbitrarily, 3). If the list is constantly changing, we can't guarantee
we return a full result to the user at any point anyways.
While here, add very basic functional tests for getmntinfo(3) to the libc
suite.
PR: 221743
Submitted by: Peter Eriksson <peter AT ifm.liu.se> (earlier version)
Sponsored by: Dell EMC Isilon
In a recent interpretation[1], "\\" shall return a non-zero value
(indicating either no match or an error).
The fix involves a change over r254091 and now the behavior matches the
Sun/IBM/HP closed source implementations and also likely musl libc.
Submitted by: Joerg Schilling <joerg at schily.net>
MFC after: 1 week
[1] http://austingroupbugs.net/view.php?id=806
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)
This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.
RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):
__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen
Reviewed by: ngie
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D11901
Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions. The functions are resolved from the rtld instead, the goal
of creating library is to avoid errors from the static linker due to
missed libdl. For static binaries, an empty .o is compiled into
libdl.a so that static binaries still get dl stubs from libc.a.
Right now lld cannot create filter objects, disable libdl on arm64
when binutils are not used.
Reviewed by: bdrewery, dim (previos version); emaste
Exp run: PR 220525, done by antoine
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D11504
Most important, use a correct signature for the
__pthread_cleanup_push_imp() stub, which was incorrectly generated
with two-args variant. The pthread_cleanup_info pointer was corrupted
in the forwarded call to the real libthr implementation, visible on
PowerPC and possibly ARM. [1]
Found and tested by: Mark Millard <markmi@dsl-only.net> [1]
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Make syslog(3) resilent to cancellation occuring in supported deferred
mode. Code must unlock syslog_mutex on cancel, install the cleanup
handler.
Diagnosed and tested by: eugen
Discussed with: dchagin
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
gets output via warnx(3)
This helps set expectations for how one might deal with those messages, i.e.,
mute output from /dev/stderr today, since that's where vwarn(3) outputs messages
to today.
MFC after: 1 month
Move an unneeded initialization, introduced in r54770 to quiet down GCC,
to a place nearer to its first use. This has no practical effect, it just
keeps the garbage better sorted.
Hinted by: OpenBSD (CVS rev. 1.56, without obfuscations)
Extend the ino_t, dev_t, nlink_t types to 64-bit ints. Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment. Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.
ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks. Unfortunately, not everything can be
fixed, especially outside the base system. For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.
Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.
Struct xvnode changed layout, no compat shims are provided.
For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.
Update note: strictly follow the instructions in UPDATING. Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.
Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver. Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem). Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).
Sponsored by: The FreeBSD Foundation (emaste, kib)
Differential revision: https://reviews.freebsd.org/D10439
This no longer seems useful. Remove it.
This was prompted by a "cast discards volatile qualifier" warning
in libthr when WARNS=6.
Reviewed by: kib
MFC after: 3 days
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10832
_local_initshells did not reset cp to the beginning of the line buffer
for every iteration that it called fgets(3), leading to writing past the
end of line with fairly long /etc/shells or excessively long line
lengths. Correct this by properly resetting cp.
PR: 192528
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: cem, jilles
Differential Revision: https://reviews.freebsd.org/D10690
Trivial style(9) fix, no functional change. There are also some 81
characters lines below, but I don't see a good way to shorten them.
Sponsored by: Dell EMC Isilon
Adapt glob's match() routine to use a greedy algorithm that avoids
exponential runtime in byzantine inputs.
While here, add a testcase for the byzantine input.
Prompted by: https://research.swtch.com/glob
Authored by: Yves Orton <demerphq at gmail.com>
Obtained from: Perl (33252c318625f3c6c89b816ee88481940e3e6f95)
Sponsored by: Dell EMC Isilon
The mutex is used in sem_open() and sem_close(), which cannot
recurse. The atfork handlers cannot collide with the open and close
code.
Reviewed by: vangyzen
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D10545
Do not retest for the found semaphore after the loop to look it up.
Instead, handle both cases of last and non-last close simultaneously,
which allows to consolidate the list unlock and successful return.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
The sysctl(HW_PAGESIZE) call cannot fail on FreeBSD kernels at least.
And even if it failed for some improbable reason, PAGE_SIZE is a safe
value to return.
Discussed with: jilles
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
The internal array size goes through a loop and is compared with numitems
which at its limits makes can be unreachably higher than arraysz.
Prevent an hypothetical overflow by matching the types.
MFC after: 1 week
When application reads large directory, calling telldir() for each entry,
like Samba does, it creates exponential performance drop as number of
entries reach tenths to hundreds of thousands. It is caused by full search
through the internal list, that never finds matches in that scenario, but
creates O(n^2) delays. This patch optimizes that search, limiting it to
entries of the same buffer, turning time closer to O(n) in case of linear
directory scan.
PR: 218622
Reviewed by: jhb, jilles
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D10408
