Literal references to /usr/local exist in a large number of files in
the FreeBSD base system. Many are in contributed software, in configuration
files, or in the documentation, but 19 uses have been identified in C
source files or headers outside the contrib and sys/contrib directories.
This commit makes it possible to set _PATH_LOCALBASE in paths.h to use
a different prefix for locally installed software.
In order to avoid changes to openssh source files, LOCALBASE is passed to
the build via Makefiles under src/secure. While _PATH_LOCALBASE could have
been used here, there is precedent in the construction of the path used to
a xauth program which depends on the LOCALBASE value passed on the compiler
command line to select a non-default directory.
This could be changed in a later commit to make the openssh build
consistently use _PATH_LOCALBASE. It is considered out-of-scope for this
commit.
Reviewed by: imp
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D26942
I noticed after the review that these shouldn't be static. Remove the
'static' from them, otherwise concurrent calls to warn* might see a
similar but to the original.
When warn() family of functions is being used after err_set_file() has
been set to, for example, /dev/null, errno is being clobbered,
rendering it unreliable after, for example, procstat_getpathname()
when it is supposed to emit a warning. Then the errno is changed to
Inappropriate ioctl for device, destroying the original value (via
calls to fprintf()functions).
Submitted by: Juraj Lutter
Differential Revision: https://reviews.freebsd.org/D26871
When building FreeBSD 11 on a FreeBSD 12 system with
CROSS_TOOLCHAIN=llvm10 we end up trying to link against the packaged
version of the sanitizer library. This resulted in a requirement for
getentropy(3) which is not present in FreeBSD 11.
Reviewed by: emaste
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D26903
- Hide ptsname_r under __BSD_VISIBLE for now as the specification
is not finalized at this time.
- Keep Symbol.map sorted.
- Avoid the interposing of ptsname_r(3) from an user application
from breaking ptsname(3) by making the implementation a static
method and call the static function from ptsname(3) instead.
Reported by: kib
Reviewed by: kib, jilles
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26845
We have to bootstrap arc4random.c, so guard the FenestrasX code to avoid
using it on Linux/macOS.
Reviewed By: cem
Differential Revision: https://reviews.freebsd.org/D26738
Push the root seed version to userspace through the VDSO page, if
the RANDOM_FENESTRASX algorithm is enabled. Otherwise, there is no
functional change. The mechanism can be disabled with
debug.fxrng_vdso_enable=0.
arc4random(3) obtains a pointer to the root seed version published by
the kernel in the shared page at allocation time. Like arc4random(9),
it maintains its own per-process copy of the seed version corresponding
to the root seed version at the time it last rekeyed. On read requests,
the process seed version is compared with the version published in the
shared page; if they do not match, arc4random(3) reseeds from the
kernel before providing generated output.
This change does not implement the FenestrasX concept of PCPU userspace
generators seeded from a per-process base generator. That change is
left for future discussion/work.
Reviewed by: kib (previous version)
Approved by: csprng (me -- only touching FXRNG here)
Differential Revision: https://reviews.freebsd.org/D22839
newlocale() optionally takes a "base" locale, from which components not
specified in the mask are inherited. POSIX says that newlocale() may
modify "base" and return it, or free "base" and return a newly allocated
locale. We were not doing either, so applications which use newlocale()
to modify an existing base locale end up leaking memory on FreeBSD.
This diff fixes the leak by releasing a reference to the base locale
before returning. This is less efficient than modifying "base"
directly, but is simpler for an initial bug fix. Also, update the man
page to clarify behaviour with respect to "base".
PR: 249416
MFC after: 3 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26522
The warning generated pre-r366207 is actually a sign comparison warning:
error: comparison of integers of different signs: 'unsigned long' and 'int'
if (strlcpy(buf, execpath, buflen) >= buflen)
Revert parts that affected other lines and just cast this to unsigned int.
The buflen < 0 -> EINVAL has been kept despite no longer serving any
purposes w.r.t. sign-extension because I do believe it's the right thing to
do: "The provided buffer was not the right size for the requested item."
The original warning is confirmed to still be gone with an:
env WARNS=6 make WITHOUT_TESTS=yes.
Reviewed by: asomers, kib
X-MFC-With: r366207
Differential Revision: https://reviews.freebsd.org/D26631
Define separate dependent targets which `afterinstallconfigs` relies on, in
order to modify `${DESTDIR}/etc/master.passwd` and
`${DESTDIR}/etc/nsswitch.conf`.
Mark these targets .PHONY, since they manipulate configurations on the fly and
the generation logic isn't 100% defined in terms of the source files/logic,
and is variable, based on MK_foo flags.
MFC after: 2 weeks
Reviewed by: bapt, brd
Differential Revision: https://reviews.freebsd.org/D20330
gdtoa wins the award for "most outdated endianness naming convention"
with its IEEE_8087 vs IEEE_MC68k defines. I had a good chuckle.
Update softfloat and arith.h to adjust to BE or LE automatically
based on the low level preprocessor defines.
Fixes printf/scanf on PowerPC64LE, although there is still a problem
lurking regarding Signalling NaNs...
Sponsored by: Tag1 Consulting, Inc.
Given that we have converted to ELFv2 for BE already, endianness is the only
difference between the two ARCHs.
As such, there is no need to differentiate LIBC_ARCH between the two.
Combining them like this lets us avoid needing to have two copies of several
bits for no good reason.
Sponsored by: Tag1 Consulting, Inc.
Sometimes nscd(8) will return a 1-byte buffer for a nonexistent entry. This
triggered an integer underflow in grp_unmarshal_func, causing getgrnam_r to
return ERANGE instead of 0.
Fix the user's buffer size check, and add a correct check for a too-small
nscd buffer.
PR: 248932
Event: September 2020 Bugathon
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Axcient
Differential Revision: https://reviews.freebsd.org/D26204
The current default is provided in various Makefile.inc in some top-level
directories and covers a good portion of the tree, but doesn't cover parts
of the build a little deeper (e.g. libcasper).
Provide a default in src.sys.mk and set WARNS to it in bsd.sys.mk if that
variable is defined. This lets us relatively cleanly provide a default WARNS
no matter where you're building in the src tree without breaking things
outside of the tree.
Crunchgen has been updated as a bootstrap tool to work on this change
because it needs r365605 at a minimum to succeed. The cleanup necessary to
successfully walk over this change on WITHOUT_CLEAN builds has been added.
There is a supplemental project to this to list all of the warnings that are
encountered when the environment has WARNS=6 NO_WERROR=yes:
https://warns.kevans.dev -- this project will hopefully eventually go away
in favor of CI doing a much better job than it.
Reviewed by: emaste, brooks, ngie (all earlier version)
Reviewed by: emaste, arichardson (depend-cleanup.sh change)
Differential Revision: https://reviews.freebsd.org/D26455
This also fixes a minor issue that was missed in the initial review; the
layout of the MFD_HUGE_* flags is actually not 1:1 bit:flag -- it instead
borrowed the Linux convention of how this is laid out since it was
originally implemented on Linux, the top 6 bits represent the shift required
for the requested page size.
This allows us to remove the flag <-> pgsize mapping table and simplify the
logic just prior to validation of the requested page size.
While we're here, fix two small nits:
- HUGETLB memfd shouldn't exhibit the SHM_GROW_ON_WRITE behavior. We can
only grow largepage shm by appropriately aligned (i.e. requested pagesize)
sizes, so it can't work in the typical/sane fashion. Furthermore, Linux
does the same, so let's be compatible.
- We don't allow MFD_HUGETLB without specifying a pagesize, so no need to
check for that later.
Reviewed by: kib (slightly earlier version)
Literally returning EINVAL from a function designed to return an fd makes
for interesting scenarios.
I cannot assign enough pointy hats to cover this one.
r365524 did accidentally invert this check that sets SHM_LARGEPAGE, leading
non-hugetlb memfd as unconfigured largepage shm and thus test failures when
we try to ftruncate or write to them.
PR: 249236
Discussed with: kib
largepage shm objects.
And since we can, add memfd_create(MFD_HUGETLB) support, hopefully
close enough to the Linux feature.
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24652
getlogin_r is specified by POSIX to to take a size_t len, not int. Fix our
version to do the same, bump the symbol version due to ABI change and
provide compat.
This was reported to break compilation of Ruby 2.8.
Some discussion about the necessity of the ABI compat did take place in the
review. While many 64-bit platforms would likely be passing it in a 64-bit
register and zero-extended and thus, not notice ABI breakage, some do
sign-extend (e.g. mips).
PR: 247102
Submitted by: Bertram Scharpf <software@bertram-scharpf.de> (original)
Submitted by: cem (ABI compat)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D26335
This is being addressed as part of a side-patch I'm working on that builds
all the things with WARNS=6, instead of relying on it being supplied in just
shallow parts of the build with higher-level Makefile.inc.
Provide a prototype for mod_main and annotate the thread function argument
as unused.
MFC after: 1 week
In a side-change that I'm working on to start defaulting src builds to
WARNS=6 where WARNS isn't otherwise specified, GCC6 (and clang, to a lesser
extent) pointed out a number of issues with the resolv tests:
- Global method variable that gets shadowed in run_tests()
- Signed/unsigned comparison between i in run_tests() and hosts->sl_cur
The shadowed variable looks like it might actually be bogus as written, as
we pass it to RUN_TESTS -> run_tests, but other parts use the global method
instead. This change is mainly geared towards correcting that by removing
the global and plumbing the method through from run_tests -> run into the
new thread.
For the signed/unsigned comparison, there's no compelling reason to not just
switch i/nthreads/nhosts to size_t.
The review also included a change to the load() function that was better
addressed by jhb in r365302.
Reviewed by: ngie, pstef
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24844
Implement the remaining pieces needed to allow userland timestamp reading.
Rewritten based on an intial essay into the problem by Justin Hibbits.
(Copyright changed to my own on his request.)
Tested on ppc64 (POWER9 Talos II), powerpcspe (e500v2 RB800), and
powerpc (g4 PowerBook).
Reviewed by: jhibbits (in irc)
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D26347
MK_MALLOC_PRODUCTION option on -CURRENT.
Also, for the sake of backwards compatibility, support the old way of
enabling 'production malloc', e.g. by adding a define in make.conf(5).
MFC after: 1 week
X-MFC-With: r365371
For historical reasons, defining MALLOC_PRODUCTION in /etc/make.conf has
been used to turn off potentially expensive debug checks and statistics
gathering in the implementation of malloc(3).
It seems more consistent to turn this into a regular src.conf(5) option,
e.g. WITH_MALLOC_PRODUCTION / WITHOUT_MALLOC_PRODUCTION. This can then
be toggled similar to any other source build option, and turned on or
off by default for e.g. stable branches.
Reviewed by: imp, #manpages
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D26337
- Use getline() instead of fgetln(). This ensures the returned string
is always null-terminated without losing the last character if the
last line in a file doesn't have a newline. Also, while fgetln says
the returned buffer can be modified, that doesn't actually seem safe
as the current implementation means you are modifying stdio's
internal buffer.
- Remove a spurious if before an ATF_REQUIRE that was clearly supposed
to be non-optional.
- Remove a pointless compare of 'ptr' against '\0' (really NULL) that
duplicated the middle condition in the for().
- Once a comment is found, skip the rest of the line, not just the
current word.
Reviewed by: kevans
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26278
CheriABI is pickier about the arguments to open(2) and crashes with a
fault if a mode isn't passed to an open() when O_CREATE is specified.
Reported by: CHERI
Reviewed by: brooks
Obtained from: CheriBSD
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26281
The size of the object at 'addr' is unknown and might be smaller than
the size of a pointer (e.g. some x86 instructions are smaller than a
pointer). Instead, just check that the address is in the bounds of
the program header.
Reported by: CHERI (indirectly)
Reviewed by: kib, brooks
Obtained from: CheriBSD
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26279
Previously this was counting the amount of spare room at the start of
the buffer that the string needed to move forward and passing that as
the number of bytes to copy to memmove rather than the length of the
string to be copied.
In the strfmon test in the test suite this caused the memmove to
overflow the allocated buffer by one byte which CHERI caught.
Reported by: CHERI
Reviewed by: kevans
Obtained from: CheriBSD
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26280
Currently we use a single bit to indicate whether the virtual page is
part of a superpage. To support a forthcoming implementation of
non-transparent 1GB superpages, it is useful to provide more detailed
information about large page sizes.
The change converts MINCORE_SUPER into a mask for MINCORE_PSIND(psind)
values, indicating a mapping of size psind, where psind is an index into
the pagesizes array returned by getpagesizes(3), which in turn comes
from the hw.pagesizes sysctl. MINCORE_PSIND(1) is equal to the old
value of MINCORE_SUPER.
For now, two bits are used to record the page size, permitting values
of MAXPAGESIZES up to 4.
Reviewed by: alc, kib
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26238
Use explicit typedefs for block thunk structures as in r264143.
Reviewed by: kib, adrian
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D26256
In most cases this simply builds the file from lib/libc for missing
functions (e.g. strlcpy on Linux etc.). In cases where this is not possible
I've added an implementation to tools/build/cross-build.
The fgetln.c/fgetwln.c/closefrom.c compatibility code was obtained from
https://gitlab.freedesktop.org/libbsd/libbsd, but I'm not sure it makes
sense to import it into to contrib just for these three bootstrap files.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D25978
Should unbreak gcc build as reported by tinderbox:
lib/libc/gen/scandir.c:59:12: warning: 'alphasort_thunk' declared 'static' but never defined [-Wunused-function]
RFC5424 defines NILVALUE as '-'. Replace its usage with a macro and
separate out the fields to be more clear. fputs(3) is used in some
places to avoid hiding possible format string problems in a macro.
Reviewed by: cem, vangyzen (earlier version)
Sponsored by: Dell EMC
Apparently it was not exported, because scandir_b.c was not included
into libc SRCS. Export it with the CURRENT-13 version.
Also, because it was not exported before ino64, clean up
scandir-compat11.c.
PR: 248572
Reported by: Alex S <iwtcex@gmail.com>
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D26020
On ELFv2, the overflow parameters in the stack frame are at a different offset
from sp than ELFv1. Adjust code to use the correct offset in all cases.
This had resulted in argv[8] and up being copied to the incorrect address
in the new context's initial stack frame.
This is not necessarily the only bug in this function, I need to do a full
review still and ensure the rest of the math is sane for ELFv2 stack frames.
Reported by: pherde (Probably. My notes are a bit unclear.)
Reviewed by: jhibbits (in irc)
Sponsored by: Tag1 Consulting, Inc.
The entire patch-set is not yet mature enough for commit, but this usable
subset is generally enough for googletest to be happy with and mostly map to
some existing concepts, so they're not as invasive.
The specific changes included here are:
- Branching in BREs with \|
- \w and \W for [[:alnum:]] and [^[:alnum:]] respectively
- \s and \S for [[:space:]] and [^[:space:]] respectively
- Additional quantifiers in BREs, \? and \+ (self-explanatory)
There's some #ifdef'd out work for allowing empty branches as a match-all.
This is a feature that's under assessment... future work will determine
how standard this behavior is and act accordingly.
The constant seems to exists on MacOS X >= 10.8.
Requested by: swills
Reviewed by: allanjude, kevans
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D25933
We bootstrap this file to allow compiling FreeBSD on Linux systems since
some boostrap tools use setmode(). Unfortunately, glibc's sys/stat.h
declares a non-static getumask() function (which is unimplemented!) and
that conflicts with the local getumask() function. To work around this
simply use a different name here.
Reviewed By: brooks, emaste
Differential Revision: https://reviews.freebsd.org/D25929
RISC-V doesn't support floating-point exceptions.
RISC-V Instruction Set Manual: Volume I: User-Level ISA, 11.2 Floating-Point
Control and Status Register: "As allowed by the standard, we do not support
traps on floating-point exceptions in the base ISA, but instead require
explicit checks of the flags in software. We considered adding branches
controlled directly by the contents of the floating-point accrued exception
flags, but ultimately chose to omit these instructions to keep the ISA simple."
We still need these functions, because some applications (notably Perl) call
them, but we cannot provide a meaningful implementation.
Sponsored by: Axiado
Differential Revision: https://reviews.freebsd.org/D25740
In IEEE 1003.1-2008 [1] and earlier revisions, BRE/ERE grammar allows for
any character to be escaped, but "ORD_CHAR preceded by an unescaped
<backslash> character [gives undefined results]".
Historically, we've interpreted an escaped ordinary character as the
ordinary character itself. This becomes problematic when some extensions
give special meanings to an otherwise ordinary character
(e.g. GNU's \b, \s, \w), meaning we may have two different valid
interpretations of the same sequence.
To make this easier to deal with and given that the standard calls this
undefined, we should throw an error (EESCAPE) if we run into this scenario
to ease transition into a state where some escaped ordinaries are blessed
with a special meaning -- it will either error out or have extended
behavior, rather than have two entirely different versions of undefined
behavior that leave the consumer of regex(3) guessing as to what behavior
will be used or leaving them with false impressions.
This change bumps the symbol version of regcomp to FBSD_1.6 and provides the
old escape semantics for legacy applications, just in case one has an older
application that would immediately turn into a pumpkin because of an
extraneous escape that's embedded or otherwise critical to its operation.
This is the final piece needed before enhancing libregex with GNU extensions
and flipping the switch on bsdgrep.
[1] http://pubs.opengroup.org/onlinepubs/9699919799.2016edition/
PR: 229925 (exp-run, courtesy of antoine)
Differential Revision: https://reviews.freebsd.org/D10510
This is neither POSIX compliant nor what the implementation does.
This could be allowed by changing the value of TCSAFLUSH from 2 to 3,
but that doesn't seem worthwhile after 25+ years.
Reviewed by: imp
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D25659
In certain situations lseek(2) will return successful although if no seek
was performed. This can happen when operating on devices that don't support
seeking (older tape drives) or when operating on changeable media devices
(such as DVD or Blu-ray devices) without a medium inserted.
Document this within the man page and update the POSIX compliance while here.
PR: 162765
Submitted by: arundel@
Reported by: arundel@
Reviewed by: bcr (mentor)
Approved by: bcr (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25646
Some older references called it 'arg'
Also fix a syntax error that was underlining an entire sentence.
PR: 247386
Reported by: Paul Floyd <paulf@free.fr>, PauAmma (research)
MFC after: 2 weeks
Sponsored by: Klara Inc.
memfd_create fds will no longer require an ftruncate(2) to set the size;
they'll grow (to the extent that it's possible) upon write(2)-like syscalls.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D25502
While this behaviour is harmless, it is really just an artifact of the
fact that the msgctl(2) implementation uses a user-visible structure as
part of the internal implementation, so it is not deliberate and these
pointers are not useful to userspace. Thus, NULL them out before
copying out, and remove references to them from the manual page.
Reported by: Jeffball <jeffball@grimm-co.com>
Reviewed by: emaste, kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25600
When the RISC-V port was initially committed to FreeBSD, GCC would
generate 64-bit long doubles, and the definitions in _fpmath.h reflected
that. This was changed to 128-bit in GCC later that year [1], but the
definitions were never updated, despite the documented workaround. This
causes printf(3) and friends to interpret only the low 64-bits of a long
double in ldtoa, thereby printing incorrect values.
Update the definitions now that both clang and GCC generate 128-bit long
doubles.
[1] 54b21fc5ae
PR: 242067
Reported by: Dennis Clarke <dclarke@blastwave.org>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25420
Posix says that the interpretation of the locale string is
"implementation-defined", so we ought to document what is
actually recognized.
Also add a cross reference to locale(1).
PR: 247553
MFC after: 1 week
Some of the NetBSD contributed tests are gated behind the
__HAVE_LONG_DOUBLE flag. This flag seems to be defined only for
platforms whose long double is larger than their double. I could not
find this explicitly documented anywhere, but it is implied by the
definitions in NetBSD's sys/arch/${arch}/include/math.h headers, and the
following assertion from the UBSAN code:
#ifdef __HAVE_LONG_DOUBLE
long double LD;
ASSERT(sizeof(LD) > sizeof(uint64_t));
#endif
RISC-V has 128-bit long doubles, so enable the tests on this platform,
and update the comments to better explain the purpose of this flag.
Reviewed by: ngie
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25419
is used by the IPPROTO_SCTP level socket options SCTP_GET_PEER_ADDRESSES
and SCTP_GET_LOCAL_ADDRESSES, which are used by libc to implement
sctp_getladdrs() and sctp_getpaddrs().
These changes allow an old libc to work on a newer kernel.
for the IPPROTO_SCTP level socket options SCTP_BINDX_ADD_ADDR and
SCTP_BINDX_REM_ADDR. These socket option are intended for internal
use only to implement sctp_bindx().
This is one user of struct sctp_getaddresses less.
struct sctp_getaddresses is strange and will be changed shortly.
- Add STANDARDS and HISTORY sections within the appropriate manpages
- Mention two USENIX papers within kqueue(2) and strlcpy(3)
Reviewed by: bcr (mentor)
Approved by: bcr (mentor)
Obtained from: NetBSD
MFC after: 7 days
Differential Revision: https://reviews.freebsd.org/D24650
libc cannot assume that aligned_alloc and free come from jemalloc, or that
any application providing its own malloc and free is actually providing
aligned_alloc.
Switch back to malloc and just make sure we're passing a properly aligned
stack into rfork_thread, as an application perhaps can't reasonably replace
just malloc or just free without headaches.
This unbreaks ksh93 after r361996, which provides malloc/free but no
aligned_alloc.
Reported by: freqlabs
Diagnosed by: Andrew Gierth <andrew_tao173.riddles.org.uk>
X-MFC-With: r361996
pthread_get_name_np() and pthread_set_name_np().
This re-applies r361770 after compatibility fixes.
Reviewed by: antoine, jkim, markj
Tested by: antoine (exp-run)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D25117
Some environments in which execvPe may be called have a limited amount of
stack available. Currently, it avoidably allocates a segment on the stack
large enough to hold PATH so that it may be mutated and use strsep() for
easy parsing. This logic is now rewritten to just operate on the immutable
string passed in and do the necessary math to extract individual paths,
since it will be copying out those segments to another buffer anyways and
piecing them together with the name for a full path.
Additional size is also needed for the stack in posix_spawnp(), because it
may need to push all of argv to the stack and rebuild the command with sh in
front of it. We'll make sure it's properly aligned for the new thread, but
future work should likely make rfork_thread a little easier to use by
ensuring proper alignment.
Some trivial cleanup has been done with a couple of error writes, moving
strings into char arrays for use with the less fragile sizeof().
Reported by: Andrew Gierth <andrew_tao173.riddles.org.uk>
Reviewed by: jilles, kib, Andrew Gierth
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25038
If execve fails with ENOEXEC, execvp is expected to rebuild the command
with /bin/sh instead and try again.
The previous version did this, but overlooked two details:
argv[0] can conceivably be NULL, in which case memp would never get
terminated. We must allocate no less than three * sizeof(char *) so we can
properly terminate at all times. For the non-NULL argv standard case, we
count all the non-NULL elements and actually skip the first argument, so we
end up capturing the NULL terminator in our bcopy().
The second detail is that the spec is actually worded such that we should
have been preserving argv[0] as passed to execvp:
"[...] executed command shall be as if the process invoked the sh utility
using execl() as follows:
execl(<shell path>, arg0, file, arg1, ..., (char *)0);
where <shell path> is an unspecified pathname for the sh utility, file is
the process image file, and for execvp(), where arg0, arg1, and so on
correspond to the values passed to execvp() in argv[0], argv[1], and so on."
So we make this change at this time as well, while we're already touching
it. We decidedly can't preserve a NULL argv[0] as this would be incredibly,
incredibly fragile, so we retain our legacy behavior of using "sh" for
argv[] in this specific instance.
Some light tests are added to try and detect some components of handling the
ENOEXEC fallback; posix_spawnp_enoexec_fallback_null_argv0 is likely not
100% reliable, but it at least won't raise false-alarms and it did result in
useful failures with pre-change libc on my machine.
This is a secondary change in D25038.
Reported by: Andrew Gierth <andrew_tao173.riddles.org.uk>
Reviewed by: jilles, kib, Andrew Gierth
MFC after: 1 week
Copying the approach chosen in r309412. This fixes building the libc tests
on a macOS host since the macOS /bin/dd binary does not support status=none.
As there only seem to be two uses, this commit changes the two Makefiles.
If this becomes more common, we could also add a wrapper bootstrap script
that ignores status= and forwards the remaining args to the real dd.
Another alternative would be to remove the status flag and pipe stderr to
/dev/null, but them we lose error messages.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D24785
This commit adds the priv(9) that waters down the sysctl to make it only
allow read(2) of a dirfd by the system root. Jailed root is not allowed, but
jail policy and superuser policy will abstain from allowing/denying it so
that a MAC module can fully control the policy.
Such a MAC module has been written, and can be found at:
https://people.freebsd.org/~kevans/mac_read_dir-0.1.0.tar.gz
It is expected that the MAC module won't be needed by many, as most only
need to do such diagnostics that require this behavior as system root
anyways. Interested parties are welcome to grab the MAC module above and
create a port or locally integrate it, and with enough support it could see
introduction to base. As noted in mac_read_dir.c, it is released under the
BSD 2 clause license and allows the restrictions to be lifted for only
jailed root or for all unprivileged users.
PR: 246412
Reviewed by: mckusick, kib, emaste, jilles, cy, phk, imp (all previous)
Reviewed by: rgrimes (latest version)
Differential Revision: https://reviews.freebsd.org/D24596
Historically, we've allowed read() of a directory and some filesystems will
accommodate (e.g. ufs/ffs, msdosfs). From the history department staffed by
Warner: <<EOF
pdp-7 unix seemed to allow reading directories, but they were weird, special
things there so I'm unsure (my pdp-7 assembler sucks).
1st Edition's sources are lost, mostly. The kernel allows it. The
reconstructed sources from 2nd or 3rd edition read it though.
V6 to V7 changed the filesystem format, and should have been a warning, but
reading directories weren't materially changed.
4.1b BSD introduced readdir because of UFS. UFS broke all directory reading
programs in 1983. ls, du, find, etc all had to be rewritten. readdir() and
friends were introduced here.
SysVr3 picked up readdir() in 1987 for the AT&T fork of Unix. SysVr4 updated
all the directory reading programs in 1988 because different filesystem
types were introduced.
In the 90s, these interfaces became completely ubiquitous as PDP-11s running
V7 faded from view and all the folks that initially started on V7 upgraded
to SysV. Linux never supported this (though I've not done the software
archeology to check) because it has always had a pathological diversity of
filesystems.
EOF
Disallowing read(2) on a directory has the side-effect of masking
application bugs from relying on other implementation's behavior
(e.g. Linux) of rejecting these with EISDIR across the board, but allowing
it has been a vector for at least one stack disclosure bug in the past[0].
By POSIX, this is implementation-defined whether read() handles directories
or not. Popular implementations have chosen to reject them, and this seems
sensible: the data you're reading from a directory is not structured in some
unified way across filesystem implementations like with readdir(2), so it is
impossible for applications to portably rely on this.
With this patch, we will reject most read(2) of a dirfd with EISDIR. Users
that know what they're doing can conscientiously set
bsd.security.allow_read_dir=1 to allow read(2) of directories, as it has
proven useful for debugging or recovery. A future commit will further limit
the sysctl to allow only the system root to read(2) directories, to make it
at least relatively safe to leave on for longer periods of time.
While we're adding logic pertaining to directory vnodes to vn_io_fault, an
additional assertion has also been added to ensure that we're not reaching
vn_io_fault with any write request on a directory vnode. Such request would
be a logical error in the kernel, and must be debugged rather than allowing
it to potentially silently error out.
Commented out shell aliases have been placed in root's chsrc/shrc to promote
awareness that grep may become noisy after this change, depending on your
usage.
A tentative MFC plan has been put together to try and make it as trivial as
possible to identify issues and collect reports; note that this will be
strongly re-evaluated. Tentatively, I will MFC this knob with the default as
it is in HEAD to improve our odds of actually getting reports. The future
priv(9) to further restrict the sysctl WILL NOT BE MERGED BACK, so the knob
will be a faithful reversion on stable/12. We will go into the merge
acknowledging that the sysctl default may be flipped back to restore
historical behavior at *any* point if it's warranted.
[0] https://www.freebsd.org/security/advisories/FreeBSD-SA-19:10.ufs.asc
PR: 246412
Reviewed by: mckusick, kib, emaste, jilles, cy, phk, imp (all previous)
Reviewed by: rgrimes (latest version)
MFC after: 1 month (note the MFC plan mentioned above)
Relnotes: absolutely, but will amend previous RELNOTES entry
Differential Revision: https://reviews.freebsd.org/D24596
One of the error descriptions referred to permissions; in context the
meaning was probably clear, but the prot values are properly called
protections.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Keep link_map l_addr binary layout compatible, rename l_addr to l_base
where rtld returns map base. Provide relocbase in newly added l_addr.
This effectively reverts the patch to the initial version of D24918.
Reported by: antoine (portmgr)
Reviewed by: jhb, markj
Tested by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24946
It previously returned the object map base address, while all other
ELF operating systems return load offset, i.e. the difference between
map base and the link base.
Explain the meaning of the field in the man page.
Stop filling the mips-only l_offs member, which is apparently unused.
PR: 246561
Requested by: Damjan Jovanovic <damjan.jov@gmail.com>
Reviewed by: emaste, jhb, cem (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D24918
functionality first appeared in FreeBSD.
Submitted by: Gordon Bergling gbergling_gmail.com
Approved by: bcr
Differential Revision: https://reviews.freebsd.org/D24677
Attempted to clean up the language around "this is a malloc'd object." May be
passed as a parameter to free(3) is a bit obtuse.
Sponsored by: Dell EMC Isilon
This option was added as a transition aide when symbol versioning was
first added. It was enabled by default in 2007 and is supported even
by the old GPLv2 binutils. Trying to disable it currently fails to
build in libc and at this point it isn't worth fixing the build.
Reported by: Michael Dexter
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D24637
sendto(2), sendmsg(2) and sendmmsg(2) return ENOTCONN if a destination
address is specified and the socket is not connected and the socket
protocol does not automatically connect ("implied connect"). Document
that. Also document the fact that the destination address is ignored
for connection-mode sockets if the socket is already connected.
PR: 245817
Submitted by: Erik Inge Bolsø <knan-bfo@modirum.com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24530
Contrary to the kevent man page, EV_EOF on a fifo is not cleared by
EV_CLEAR. Modify the read and write filters to clear EV_EOF when the
fifo's PIPE_EOF flag is clear, and update the man page to document the
new behaviour.
Modify the write filter to return the amount of buffer space available
even if no readers are present. This matches the behaviour for sockets.
When reading from a pipe, only call pipeselwakeup() if some data was
actually read. This prevents the continuous re-triggering of a
EVFILT_READ event on EOF when in edge-triggered mode.
PR: 203366, 224615
Submitted by: Jan Kokemüller <jan.kokemueller@gmail.com>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D24528
In r326576 ("use @@@ instead of @@ in __sym_default"), an earlier version of
the phabricator-discussed patch was inadvertently committed. The commit
message claims that @@@ means that weak is not needed, but that was due to a
misunderstanding of the use of weak symbols in this context by the submitted
in the first draft of the patch; the description text was not updated to
match the discussion. As discussed in phabricator, weak is needed for
symbol interposing because of the behavior of our rtld, and is widely used
elsewhere in libc.
This partial revert restores the approved version of the patch and permits
symbol interposing for openat.
Reported by: Raymond Ramsden <rramsden AT isilon.com>
Reviewed by: dim, emaste, kib (2017)
Discussed with: kib (2020)
Differential Revision: https://reviews.freebsd.org/D11653
Revert r354606 to restore r354605.
Apply one line from jemalloc commit d01b425e5d1e1 in hash_x86_128()
to fix the build with gcc, which only allows a fallthrough attribute
to appear before a case or default label.
Submitted by: jasone in r354605
Discussed with: jasone
Reviewed by: bdrewery
MFC after: never, due to gcc 4.2.1
Relnotes: yes
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D24522
If the index we're trying to convert is 0 we can avoid a potentially
expensive call to getifaddrs(). No interface has an ifindex of zero, so
we can handle this as an error: set the errno to ENXIO and return NULL.
Submitted by: Nick Rogers
Reviewed by: lutz at donnerhacke.de
MFC after: 2 weeks
Sponsored by: RG Nets
Differential Revision: https://reviews.freebsd.org/D24524
In the below-referenced PR, a case is attached of a simple reproducer that
exhibits suboptimal behavior: EVFILT_READ and EVFILT_WRITE being set in the
same kevent(2) call will only honor the first one. This is, in-fact, how
it's supposed to work.
A read of the manpage leads me to believe we could be more clear about this;
right now there's a logical leap to make in the relevant statement: "When
passed as input, it forces EV_ERROR to always be returned." -- the logical
leap being that this indicates the caller should have allocated space for
the change to be returned with EV_ERROR indicated in the events, or
subsequent filters will get dropped on the floor.
Another possible workaround that accomplishes similar effect without needing
space for all events is just setting EV_RECEIPT on the final change being
passed in; if any errored before it, the kqueue would not be drained. If we
made it to the final change with EV_RECEIPT set, then we would return that
one with EV_ERROR and still not drain the kqueue. This would seem to not be
all that advisable.
PR: 229741
MFC after: 1 week
This avoids passing garbage to sigprocmask() if the jump buffer is
invalid.
Reviewed by: mhorne
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24483
This saves a system call and avoids one of the (relatively rare) cases
of the kernel exporting pointers via sysctl.
As a temporary measure, keep the sysctl support to allow limited
compatability with old kernels.
Fail gracefully if ps_strings can't be found (should never happen).
Reviewed by: kib
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24407
This will be used by setproctitle().
Reviewed by: kib
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24407
This pattern is used in callbacks with void * data arguments and seems
both relatively uncommon and relatively harmless. Silence the warning
by casting through uintptr_t.
This warning is on by default in Clang 11.
Reviewed by: arichardson
Obtained from: CheriBSD (partial)
MFC after: 1 week
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24425
Include a temporarily compatibility shim as well for kernels predating
close_range, since closefrom is used in some critical areas.
Reviewed by: markj (previous version), kib
Differential Revision: https://reviews.freebsd.org/D24399
sonewconn() emits debug-level messages when a listen socket's queue
overflows. Currently, sonewconn() tracks overflows on a global basis. It
will only log one message every 60 seconds, regardless of how many sockets
experience overflows. And, when it next logs at the end of the 60 seconds,
it records a single message referencing a single PCB with the total number
of overflows across all sockets.
This commit changes to per-socket overflow tracking. The code will now
log one message every 60 seconds per socket. And, the code will provide
per-socket queue length and overflow counts. It also provides a way to
change the period between log messages using a sysctl.
Reviewed by: jhb (previous version), bcr (manpages)
MFC after: 2 weeks
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D24316
This had been introduced to ease any pain for using slightly older kernels
with a newer libc, e.g., for bisecting a kernel across the introduction of
shm_open2(2). 6 months has passed, retire the fallback and let shm_open()
unconditionally call shm_open2().
Stale includes are removed as well.
arc4random.c: In the incredibly unbelievable circumstance where
_rs_init() fails to allocate pages, don't call abort() because of
corefile data leakage concerns, but simply _exit(). The reasoning
is _rs_init() will only fail if someone finds a way to apply
specific pressure against this failure point, for the purpose of
leaking information into a core which they can read. We don't
need a corefile in this instance to debug that. So take this
"lever" away from whoever in the future wants to do that.
arc4random.3: reference random(4)
arc4random_uniform.c: include stdint.h over sys/types.h
close_range(min, max, flags) allows for a range of descriptors to be
closed. The Python folk have indicated that they would much prefer this
interface to closefrom(2), as the case may be that they/someone have special
fds dup'd to higher in the range and they can't necessarily closefrom(min)
because they don't want to hit the upper range, but relocating them to lower
isn't necessarily feasible.
sys_closefrom has been rewritten to use kern_close_range() using ~0U to
indicate closing to the end of the range. This was chosen rather than
requiring callers of kern_close_range() to hold FILEDESC_SLOCK across the
call to kern_close_range for simplicity.
The flags argument of close_range(2) is currently unused, so any flags set
is currently EINVAL. It was added to the interface in Linux so that future
flags could be added for, e.g., "halt on first error" and things of this
nature.
This patch is based on a syscall of the same design that is expected to be
merged into Linux.
Reviewed by: kib, markj, vangyzen (all slightly earlier revisions)
Differential Revision: https://reviews.freebsd.org/D21627
This is the foundational change for the routing subsytem rearchitecture.
More details and goals are available in https://reviews.freebsd.org/D24141 .
This patch introduces concept of nexthop objects and new nexthop-based
routing KPI.
Nexthops are objects, containing all necessary information for performing
the packet output decision. Output interface, mtu, flags, gw address goes
there. For most of the cases, these objects will serve the same role as
the struct rtentry is currently serving.
Typically there will be low tens of such objects for the router even with
multiple BGP full-views, as these objects will be shared between routing
entries. This allows to store more information in the nexthop.
New KPI:
struct nhop_object *fib4_lookup(uint32_t fibnum, struct in_addr dst,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
struct nhop_object *fib6_lookup(uint32_t fibnum, const struct in6_addr *dst6,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
These 2 function are intended to replace all all flavours of
<in_|in6_>rtalloc[1]<_ign><_fib>, mpath functions and the previous
fib[46]-generation functions.
Upon successful lookup, they return nexthop object which is guaranteed to
exist within current NET_EPOCH. If longer lifetime is desired, one can
specify NHR_REF as a flag and get a referenced version of the nexthop.
Reference semantic closely resembles rtentry one, allowing sed-style conversion.
Additionally, another 2 functions are introduced to support uRPF functionality
inside variety of our firewalls. Their primary goal is to hide the multipath
implementation details inside the routing subsystem, greatly simplifying
firewalls implementation:
int fib4_lookup_urpf(uint32_t fibnum, struct in_addr dst, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
int fib6_lookup_urpf(uint32_t fibnum, const struct in6_addr *dst6, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
All functions have a separate scopeid argument, paving way to eliminating IPv6 scope
embedding and allowing to support IPv4 link-locals in the future.
Structure changes:
* rtentry gets new 'rt_nhop' pointer, slightly growing the overall size.
* rib_head gets new 'rnh_preadd' callback pointer, slightly growing overall sz.
Old KPI:
During the transition state old and new KPI will coexists. As there are another 4-5
decent-sized conversion patches, it will probably take a couple of weeks.
To support both KPIs, fields not required by the new KPI (most of rtentry) has to be
kept, resulting in the temporary size increase.
Once conversion is finished, rtentry will notably shrink.
More details:
* architectural overview: https://reviews.freebsd.org/D24141
* list of the next changes: https://reviews.freebsd.org/D24232
Reviewed by: ae,glebius(initial version)
Differential Revision: https://reviews.freebsd.org/D24232
This is an application of the kernel overflow fix from r357948 to
userspace, based on the algorithm developed by Bruce Evans. To keep
the ABI of the vds_timekeep stable, instead of adding the large_delta
member, MSB of both multipliers are added to quickly estimate the overflow.
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
instead of sprinkling them out over many disjoint files. This is a follow-up
to achieve the same goal in an incomplete rev.348521.
Approved by: imp
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D20520
These functions first appeared in the First Edition of Unix (or earlier in the
pdp-7 version). Just claim 1st Edition for all this. The pdp-7 code is too
fragmented at this point to extend history that far back.
While here, drop an extra conjunction from the list of error
conditions for the remaining EIO description in symlink(2).
Discussed with: mckusick (trimming duplicates)
MFC after: 2 weeks
EINTEGRITY was previously documented as a UFS-specific error for
mount(2). This documents EINTEGRITY as a filesystem-independent error
that may be reported by the backing store of a filesystem.
While here, document EIO as a filesystem-independent error for both
mount(2) and posix_fadvise(2). EIO was previously only documented for
UFS for mount(2).
Reviewed by: mckusick
Suggested by: mckusick
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D24168
For this, things are complicated. The first mention in the manual was in the 4th
edition manual (as an add on to exec(II)). The 2nd and 3rd editions didn't have
these in the manual (either as a separate page, or as an add-on to exec(II)). We
don't have good 1st, 2nd or 3rd edition distributions to look in. However,
there's a tape labeled 'last1120c' that we do have. This tape contains the last
version of the V2 edition of the C compiler on it (just after C got struct). On
this tape there was a libc.sa archive that contains source for execl and
execp. This source is sufficiently different from the V5 sources (which are the
next ones we have sources for) and have a slightly different calling convention
than later sources, suggesting that the early date for the last1120c tape is
correct (in that era, the epoch changed every year, leading to a one or two year
ambiguity on when the files could have been modified) and it should be though of
as V2. Since this was also a time of compiler development, and the calling
convetions are known to be under evolution, and since the rest of the sources in
libc.sa are consistent, that's further evidence that V2 is likely. Finally, 2nd
edition was the last version to fully support the 11/20 because it lacked many
basic features and bell labs moved off it to the 11/45 as soon as they could
afford to buy one, around this time era. The unix manuals make it sound like V3
might have supported the 11/20, but the same intro could also be read to mean it
didn't, at all, and that V3 was the first rewrite for the 11/45 ahead of the
rewrite in C that came with V4.
Taken together, the evidence leans most heavily to V2 (90% IMHO), and slightly
to V3 (8%) or possibly V4 (2%). I've not put all this in the man page, but have
left it here in case someone notices in the future that V4 is the first manual
page for it.
In a single-threaded program pthread_getspecific() always returns NULL,
so the old locale would not end up being freed.
PR: 239520
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
The current code uses a rwlock to protect the cached list, which
in turn holds a list of catentry objects, and increments reference
count while holding only read lock.
Fix this by converting the reference counter to use atomic operations.
While I'm there, also perform some clean ups around memory operations.
PR: 202636
Reported by: Henry Hu <henry.hu.sh@gmail.com>
Reviewed by: markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D24095
Crash was noticed by pkubaj building gcc9.
Apparently non dword-aligned char pointers are somewhat rare in the wild.
Reported by: pkubaj
Sponsored by: Tag1 Consulting, Inc.
