Currently when an [EAGAIN] is encountered we return a partial result
that does not contain the delimeter. On the next (successful) read we
were returning the next part of the line without the preceding string
from the first failed call.
Fix this by using the same mechanism as ungetc(3) does. For the buffered
case we could simply set fp->_r and fp->_p back to their values before
sappend() is ran but for simplicity ungetc(3) is done in there as well.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D31687
Instead of polling nleft[i] (without appropriate memory barriers!) and
using sleep() to detect the exit just call pthread_join() on all threads.
Also replace the use of a mutex that guarding the increments with atomic
fetch_add. This should reduce the runtime of this test on SMP systems.
Finally, remove all the debug printfs unless DEBUG_OUTPUT is set in
the environment.
Test Plan: still fails sometimes on qemu (but maybe less often?)
Reviewed By: jhb
Differential Revision: https://reviews.freebsd.org/D29390
With an out-of-tree Clang, we can use the -resource-dir flag when linking
to point it at the runtime libraries from the current SYSROOT.
This moves the path to the clang-internal library directory to a separate
.mk file that can be used by Makefiles that want to find the sanitizer
libraries. I intend to re-use this .mk file for my upcoming changes that
allow building the entire base system with ASAN/UBSAN/MSAN.
Reviewed By: dim
Differential Revision: https://reviews.freebsd.org/D28852
The default behavior for attaching processes to jails is that the jail's
cpuset augments the attaching processes, so that it cannot be used to
escalate a user's ability to take advantage of more CPUs than the
administrator wanted them to.
This is problematic when root needs to manage jails that have disjoint
sets with whatever process is attaching, as this would otherwise result
in a deadlock. Therefore, if we did not have an appropriate common
subset of cpus/domains for our new policy, we now allow the process to
simply take on the jail set *if* it has the privilege to widen its mask
anyways.
With the new logic, root can still usefully cpuset a process that
attaches to a jail with the desire of maintaining the set it was given
pre-attachment while still retaining the ability to manage child jails
without jumping through hoops.
A test has been added to demonstrate the issue; cpuset of a process
down to just the first CPU and attempting to attach to a jail without
access to any of the same CPUs previously resulted in EDEADLK and now
results in taking on the jail's mask for privileged users.
PR: 253724
Reviewed by: jamie (also discussed with)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D28952
This flag has been set on startup since 65618fdda0.
However, This causes some of the math-related tests to fail as they report
zero instead of a tiny number. This fixes at least
/usr/tests/lib/msun/ldexp_test and possibly others.
Additionally, setting this flag prevents printf() from printing subnormal
numbers in decimal form.
See also https://www.openwall.com/lists/musl/2021/02/26/1
PR: 253847
Reviewed By: mmel
Differential Revision: https://reviews.freebsd.org/D28938
Some NSS regression tests for getgrent(3) and getpwent(3) were not
testing anything because the test incorrectly requested creation of a
database snapshot.
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
As suggested in D27598. This also supports MK_WERROR.clang=no and
MK_WERROR.gcc=no to support the existing NO_WERROR.<compiler> uses.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D27601
The cpuset(2) tests should be run as root (require.user properly set) with
>= 3 cpus for maximum coverage. All tests that want to modify the cpuset
don't assume any particular cpu layout (i.e. the first cpu may not be 0, the
last may not be first + count) and the following scenarios are tested:
1.) newset: basic execute cpuset() to grab a new cpuset, make sure the
assigned cpuset then has a different ID.
2.) transient: create a new cpuset then assign the process its original
cpuset, ensuring that the one we created is now gone.
3.) deadlk: test assigning an anonymous mask, then resetting the process
base affinity with 1-cpu overlap w.r.t. the anonymous mask and with
0-cpu overlap w.r.t. the anonymous mask.
4.) jail_attach_newbase: process attaches to a jail with its own
cpuset+mask (e.g. cpuset -c -l 1,2 jail -c path=/ command=/bin/sh)
5.) jail_attach_newbase_plain: process attaches to a jail with its own
cpuset (e.g. cpuset -c jail -c path=/ command=/bin/sh)
6.) jail_attach_prevbase: process attaches to a jail with the containing
jail's root cpuset (e.g. jail -c path=/ command=/bin/sh)
7.) jail_attach_plain: process attaches to a jail with the containing jail's
root cpuset+mask.
8.) badparent: creates a new cpuset and modifies the anonymous thread mask,
then setid's back to the original and checks that cpuset_getid() returns
the expected set.
Differential Revision: https://reviews.freebsd.org/D27307
Add shims to map NetBSD's API to CPUSET(9). Obviously the invalid input
parts of these tests are relatively useless since we're just testing the
shims that aren't used elsewhere, there's still some amount of value in
the parts testing valid inputs.
Differential Revision: https://reviews.freebsd.org/D27307
r366981 disabled ASAN when it might not be reliable (with an external
compiler), but this test is broken without ASAN so disable it completely
in that case.
PR: 250706
Reviewed by: emaste, lwhsu
Differential Revision: https://reviews.freebsd.org/D26982
When building FreeBSD 11 on a FreeBSD 12 system with
CROSS_TOOLCHAIN=llvm10 we end up trying to link against the packaged
version of the sanitizer library. This resulted in a requirement for
getentropy(3) which is not present in FreeBSD 11.
Reviewed by: emaste
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D26903
The current default is provided in various Makefile.inc in some top-level
directories and covers a good portion of the tree, but doesn't cover parts
of the build a little deeper (e.g. libcasper).
Provide a default in src.sys.mk and set WARNS to it in bsd.sys.mk if that
variable is defined. This lets us relatively cleanly provide a default WARNS
no matter where you're building in the src tree without breaking things
outside of the tree.
Crunchgen has been updated as a bootstrap tool to work on this change
because it needs r365605 at a minimum to succeed. The cleanup necessary to
successfully walk over this change on WITHOUT_CLEAN builds has been added.
There is a supplemental project to this to list all of the warnings that are
encountered when the environment has WARNS=6 NO_WERROR=yes:
https://warns.kevans.dev -- this project will hopefully eventually go away
in favor of CI doing a much better job than it.
Reviewed by: emaste, brooks, ngie (all earlier version)
Reviewed by: emaste, arichardson (depend-cleanup.sh change)
Differential Revision: https://reviews.freebsd.org/D26455
This is being addressed as part of a side-patch I'm working on that builds
all the things with WARNS=6, instead of relying on it being supplied in just
shallow parts of the build with higher-level Makefile.inc.
Provide a prototype for mod_main and annotate the thread function argument
as unused.
MFC after: 1 week
In a side-change that I'm working on to start defaulting src builds to
WARNS=6 where WARNS isn't otherwise specified, GCC6 (and clang, to a lesser
extent) pointed out a number of issues with the resolv tests:
- Global method variable that gets shadowed in run_tests()
- Signed/unsigned comparison between i in run_tests() and hosts->sl_cur
The shadowed variable looks like it might actually be bogus as written, as
we pass it to RUN_TESTS -> run_tests, but other parts use the global method
instead. This change is mainly geared towards correcting that by removing
the global and plumbing the method through from run_tests -> run into the
new thread.
For the signed/unsigned comparison, there's no compelling reason to not just
switch i/nthreads/nhosts to size_t.
The review also included a change to the load() function that was better
addressed by jhb in r365302.
Reviewed by: ngie, pstef
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24844
- Use getline() instead of fgetln(). This ensures the returned string
is always null-terminated without losing the last character if the
last line in a file doesn't have a newline. Also, while fgetln says
the returned buffer can be modified, that doesn't actually seem safe
as the current implementation means you are modifying stdio's
internal buffer.
- Remove a spurious if before an ATF_REQUIRE that was clearly supposed
to be non-optional.
- Remove a pointless compare of 'ptr' against '\0' (really NULL) that
duplicated the middle condition in the for().
- Once a comment is found, skip the rest of the line, not just the
current word.
Reviewed by: kevans
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26278
CheriABI is pickier about the arguments to open(2) and crashes with a
fault if a mode isn't passed to an open() when O_CREATE is specified.
Reported by: CHERI
Reviewed by: brooks
Obtained from: CheriBSD
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26281
Some of the NetBSD contributed tests are gated behind the
__HAVE_LONG_DOUBLE flag. This flag seems to be defined only for
platforms whose long double is larger than their double. I could not
find this explicitly documented anywhere, but it is implied by the
definitions in NetBSD's sys/arch/${arch}/include/math.h headers, and the
following assertion from the UBSAN code:
#ifdef __HAVE_LONG_DOUBLE
long double LD;
ASSERT(sizeof(LD) > sizeof(uint64_t));
#endif
RISC-V has 128-bit long doubles, so enable the tests on this platform,
and update the comments to better explain the purpose of this flag.
Reviewed by: ngie
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25419
If execve fails with ENOEXEC, execvp is expected to rebuild the command
with /bin/sh instead and try again.
The previous version did this, but overlooked two details:
argv[0] can conceivably be NULL, in which case memp would never get
terminated. We must allocate no less than three * sizeof(char *) so we can
properly terminate at all times. For the non-NULL argv standard case, we
count all the non-NULL elements and actually skip the first argument, so we
end up capturing the NULL terminator in our bcopy().
The second detail is that the spec is actually worded such that we should
have been preserving argv[0] as passed to execvp:
"[...] executed command shall be as if the process invoked the sh utility
using execl() as follows:
execl(<shell path>, arg0, file, arg1, ..., (char *)0);
where <shell path> is an unspecified pathname for the sh utility, file is
the process image file, and for execvp(), where arg0, arg1, and so on
correspond to the values passed to execvp() in argv[0], argv[1], and so on."
So we make this change at this time as well, while we're already touching
it. We decidedly can't preserve a NULL argv[0] as this would be incredibly,
incredibly fragile, so we retain our legacy behavior of using "sh" for
argv[] in this specific instance.
Some light tests are added to try and detect some components of handling the
ENOEXEC fallback; posix_spawnp_enoexec_fallback_null_argv0 is likely not
100% reliable, but it at least won't raise false-alarms and it did result in
useful failures with pre-change libc on my machine.
This is a secondary change in D25038.
Reported by: Andrew Gierth <andrew_tao173.riddles.org.uk>
Reviewed by: jilles, kib, Andrew Gierth
MFC after: 1 week
Copying the approach chosen in r309412. This fixes building the libc tests
on a macOS host since the macOS /bin/dd binary does not support status=none.
As there only seem to be two uses, this commit changes the two Makefiles.
If this becomes more common, we could also add a wrapper bootstrap script
that ignores status= and forwards the remaining args to the real dd.
Another alternative would be to remove the status flag and pipe stderr to
/dev/null, but them we lose error messages.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D24785
O_SEARCH is defined by POSIX [0] to open a directory for searching, skipping
permissions checks on the directory itself after the initial open(). This is
close to the semantics we've historically applied for O_EXEC on a directory,
which is UB according to POSIX. Conveniently, O_SEARCH on a file is also
explicitly undefined behavior according to POSIX, so O_EXEC would be a fine
choice. The spec goes on to state that O_SEARCH and O_EXEC need not be
distinct values, but they're not defined to be the same value.
This was pointed out as an incompatibility with other systems that had made
its way into libarchive, which had assumed that O_EXEC was an alias for
O_SEARCH.
This defines compatibility O_SEARCH/FSEARCH (equivalent to O_EXEC and FEXEC
respectively) and expands our UB for O_EXEC on a directory. O_EXEC on a
directory is checked in vn_open_vnode already, so for completeness we add a
NOEXECCHECK when O_SEARCH has been specified on the top-level fd and do not
re-check that when descending in namei.
[0] https://pubs.opengroup.org/onlinepubs/9699919799/
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D23247
to port software written for Linux variant of qsort_r(3).
Reviewed by: kib, arichardson
MFC after: 2 weeks
Relnotes: yes
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23174
These functions (sigandset, sigisemptyset, sigorset) are commonly available
in at least musl libc and glibc; sigorset, at least, has proven quite useful
in qemu-bsd-user work for tracking the current process signal mask in a more
self-documenting/aesthetically pleasing manner.
Reviewed by: bapt, jilles, pfg
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22187
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22494
This warning (comparing a pointer against a zero character literal
rather than NULL) has existed since GCC 7.1.0, and was recently added to
Clang trunk.
Almost all of these are harmless, except for fwcontrol's str2node, which
needs to both guard against dereferencing a NULL pointer (though in
practice it appears none of the callers will ever pass one in), as well
as ensure it doesn't parse the empty string as node 0 due to strtol's
awkward interface.
Submitted by: James Clarke <jtrc27@jrtc27.com>
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D21914
This unskips:
- lib.libc.stdlib.strtod_test.strtod_round
- lib.msun.fe_round_test.t_nofe_round
In lib/msun/tests/Makefile only define on fe_round_test.c because
lib.msun.ilogb_test.ilogb will get wrong results and needs more examination.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
This avoids reading past the end of the static strings. On a system
with bounds checking these tests fault.
Reviewed by: asomers
Obtained from: CheriBSD
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D21004
Historically we have not distinguished between kernel wirings and user
wirings for accounting purposes. User wirings (via mlock(2)) were
subject to a global limit on the number of wired pages, so if large
swaths of physical memory were wired by the kernel, as happens with
the ZFS ARC among other things, the limit could be exceeded, causing
user wirings to fail.
The change adds a new counter, v_user_wire_count, which counts the
number of virtual pages wired by user processes via mlock(2) and
mlockall(2). Only user-wired pages are subject to the system-wide
limit which helps provide some safety against deadlocks. In
particular, while sources of kernel wirings typically support some
backpressure mechanism, there is no way to reclaim user-wired pages
shorting of killing the wiring process. The limit is exported as
vm.max_user_wired, renamed from vm.max_wired, and changed from u_int
to u_long.
The choice to count virtual user-wired pages rather than physical
pages was done for simplicity. There are mechanisms that can cause
user-wired mappings to be destroyed while maintaining a wiring of
the backing physical page; these make it difficult to accurately
track user wirings at the physical page layer.
The change also closes some holes which allowed user wirings to succeed
even when they would cause the system limit to be exceeded. For
instance, mmap() may now fail with ENOMEM in a process that has called
mlockall(MCL_FUTURE) if the new mapping would cause the user wiring
limit to be exceeded.
Note that bhyve -S is subject to the user wiring limit, which defaults
to 1/3 of physical RAM. Users that wish to exceed the limit must tune
vm.max_user_wired.
Reviewed by: kib, ngie (mlock() test changes)
Tested by: pho (earlier version)
MFC after: 45 days
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D19908
Initialize `oldlen` to the size of the value, instead of leaving the value
unitialized. Leaving it unitialized seems to work by accident on amd64 when
running 64-bit programs, but not on i386.
This matches patterns in use in other programs.
PR: 237458
Approved by: emaste (mentor; implicit)
MFC after: 1 week
Tested on: ^/head (amd64), ^/stable/11 (i386)
The current logic for CSTD/CXXSTD requires homogenity as far as the
supported C/C++ standards, which is a sensible default. However, when
dealing with differing versions of C++, some code may compile with C++11, but
not C++17 (for instance). So in order to avoid having people convert over their
code to the new standard, give the users the ability to specify the standard on
a per-program basis.
This will allow a user to override the supporting standard for a set of
programs, mixing C++11 with C++14 (for instance).
Reviewed by: asomers
Apprved by: emaste (mentor)
MFC after: 1 month
MFC with: r345708
Differential Revision: https://reviews.freebsd.org/D19738
When a review is closed via Phabricator it updates the patch attached to the
review. I downloaded the raw patch from Phabricator, applied it, and repeated
my mistake from r345704 by accident mixing content from D19732 and D19738.
For my own personal sanity, I will try not to mix reviews like this in the
future.
MFC after: 1 month
MFC with: r345706
Approved by: emaste (mentor, implicit)
CXXSTD was added as the C++ analogue to CSTD.
CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.
This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.
As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.
Notes:
This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.
Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:
Before this commit:
```
CXXFLAGS+= -std=c++14
```
After this commit:
```
CXXSTD= c++14
```
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 month
MFC with: r345203, r345704, r345705
Relnotes: yes
Tested with: make tinderbox
Differential Revision: https://reviews.freebsd.org/D19732
I accidentally committed code from two reviews. I will reintroduce the code to
bsd.progs.mk as part of a separate commit from r345704.
Approved by: emaste (mentor, implicit)
MFC after: 2 months
MFC with: r345704
CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.
This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.
As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.
Notes:
This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.
Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:
Before this commit:
```
CXXFLAGS+= -std=c++14
```
After this commit:
```
CXXSTD= c++14
```
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 month
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D19732
In short, the prior code was far too simplistic when it came to calling recv(2)
and failed intermittently (or in the case of Jenkins, deterministically).
Handle short recv(2)s by checking the return code and incrementing the window
into the buffer by the number of received bytes. If the number of received
bytes <= 0, then bail out of the loop, and test the total number of received
bytes vs the expected number of bytes sent for equality, and base whether or
not the test passes/fails on that fact.
Remove the expected failure, now that the hdtr testcases deterministically pass
on my host after this change [1].
PR: 234809 [1], 235200
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19188
ci.FreeBSD.org does not have access to a DNS resolver/network (unlike my test
VM), so in order for the test to pass on the host, it needs to avoid the DNS
lookup by using the numeric host address representation.
PR: 235200
Reviewed by: asomers, lwhsu
Approved by: emaste (mentor)
MFC after: 2 weeks
MFC with: r343362, r343365, r343367-r343368, r343461
Differential Revision: https://reviews.freebsd.org/D19026
