In short, the prior code was far too simplistic when it came to calling recv(2)
and failed intermittently (or in the case of Jenkins, deterministically).
Handle short recv(2)s by checking the return code and incrementing the window
into the buffer by the number of received bytes. If the number of received
bytes <= 0, then bail out of the loop, and test the total number of received
bytes vs the expected number of bytes sent for equality, and base whether or
not the test passes/fails on that fact.
Remove the expected failure, now that the hdtr testcases deterministically pass
on my host after this change [1].
PR: 234809 [1], 235200
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19188
ci.FreeBSD.org does not have access to a DNS resolver/network (unlike my test
VM), so in order for the test to pass on the host, it needs to avoid the DNS
lookup by using the numeric host address representation.
PR: 235200
Reviewed by: asomers, lwhsu
Approved by: emaste (mentor)
MFC after: 2 weeks
MFC with: r343362, r343365, r343367-r343368, r343461
Differential Revision: https://reviews.freebsd.org/D19026
The return value (`err`) should be checked; not the `errno` value.
PR: 235200
Approved by: emaste (mentor)
Reviewed by: asomers, lwhsu
MFC after: 28 days
MFC with: r343362, r343365, r343367-r343368
Differential Revision: https://reviews.freebsd.org/D18969
I should have only changed the format qualifier with the `size_t` value,
`length`, not the other [`off_t`] value, `dest_file_size`.
MFC after: 1 month
MFC with: r343362, r343365, r343367
Approved by: emaste (mentor; implicit)
Reported by: gcc 8.x
gcc 8.x is more pedantic than clang 7.x with format strings and the tests
passed `void*` variables while supplying `%s` (which is technically
incorrect).
Make the affected `void*` variables use `char*` storage instead to address
this issue, as the compiler will upcast the values to `char*`.
MFC after: 1 month
MFC with: r343362
Approved by: emaste (mentor; implicit)
Reviewed by: asomers
Differential Revision: https://reviews.freebsd.org/D18934
These testcases exercise a number of functional requirements for sendfile(2).
The testcases use IPv4 and IPv6 domain sockets with TCP, and were confirmed
functional on UFS and ZFS. UDP address family sockets cannot be used per the
sendfile(2) contract, thus using UDP sockets is outside the scope of
testing the syscall in positive cases. As seen in
`:s_negative_udp_socket_test`, UDP is used to test the sendfile(2) contract
to ensure that EINVAL is returned by sendfile(2).
The testcases added explicitly avoid testing out `SF_SYNC` due to the
complexity of verifying that support. However, this is a good next logical
item to verify.
The `hdtr_positive*` testcases work to a certain degree (the header
testcases pass), but the trailer testcases do not work (it is an expected
failure). In particular, the value received by the mock server doesn't match
the expected value, and instead looks something like the following (using
python array notation):
`trailer[:]message[1:]`
instead of:
`message[:]trailer[:]`
This makes me think there's a buffer overrun issue or problem with the
offset somewhere in the sendfile(2) system call, but I need to do some
other testing first to verify that the code is indeed sane, and my
assumptions/code isn't buggy.
The `sbytes_negative` testcases that check `sbytes` being set to an
invalid value resulting in `EFAULT` fails today as the other change
(which checks `copyout(9)`) has not been committed [1]. Thus, it
should remain an expected failure (see bug 232210 for more details
on this item).
Next steps for testing sendfile(2):
1. Fix the header/trailer testcases so that they pass.
2. Setup if_tap interface and test with it, instead of using "localhost", per
@asomers's suggestion.
3. Handle short recv(2)'s in `server_cat(..)`.
4. Add `SF_SYNC` support.
5. Add some more negative tests outside the scope of the functional contract.
MFC after: 1 month
Reviewed by: asomers
Approved by: emaste (mentor)
PR: 232210
Sponsored by: Netflix, Inc
Differential Revision: https://reviews.freebsd.org/D18625
This fixes the obscure endless loop seen with case-insensitive
patterns containing characters in 128-255 range; originally
found running GNU grep test suite.
Our regex implementation being kludgy translates the characters
in case-insensitive pattern to bracket expression containing both
cases for the character and doesn't correctly handle the case when
original character is in bitmap and the other case is not, falling
into the endless loop going through in p_bracket(), ordinary(),
and bothcases().
Reducing the bitmap to 0-127 range for multibyte locales solves this
as none of these characters have other case mapping outside of bitmap.
We are also safe in the case when the original character outside of
bitmap has other case mapping in the bitmap (there are several of those
in our current ctype maps having unidirectional mapping into bitmap).
Reviewed by: bapt, kevans, pfg
Differential revision: https://reviews.freebsd.org/D18302
Matcher function incorrectly assumed that moffset that we get from
findmust is in bytes. Fix this by introducing a stepback function,
taking short path if MB_CUR_MAX is 1, and going back byte-by-byte,
checking if we have a legal character sequence otherwise.
PR: 153502
Reviewed by: pfg, kevans
Approved by: kib (mentor, implicit)
Differential revision: https://reviews.freebsd.org/D18297
Currently libclang_rt is not provided for cross-building and as such
is not connected to cross-tools. For building clang once in universe
it is likely that libclang_rt won't exist for the universe toolchain
but even if it did it would not support anything but the native arch.
So explicitly check for support before enabling h_raw.
MFC after: 1 week
Reviewed by: dim
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D16012
Previously, libc.so would initialize its notion of the break address
using _end, a special symbol emitted by the static linker following
the bss section. Compatibility issues between lld and ld.bfd could
cause the wrong definition of _end (libc.so's definition rather than
that of the executable) to be used, breaking the brk()/sbrk()
interface.
Avoid this problem and future interoperability issues by simply not
relying on _end. Instead, modify the break() system call to return
the kernel's view of the current break address, and have libc
initialize its state using an extra syscall upon the first use of the
interface. As a side effect, this appears to fix brk()/sbrk() usage
in executables run with rtld direct exec, since the kernel and libc.so
no longer maintain separate views of the process' break address.
PR: 228574
Reviewed by: kib (previous version)
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D15663
On older kernels, when userspace program disables SIGSYS, catch ENOSYS and
emulate getrandom(2) syscall with the kern.arandom sysctl (via existing
arc4_sysctl wrapper).
Special care is taken to faithfully emulate EFAULT on NULL pointers, because
sysctl(3) as used by kern.arandom ignores NULL oldp. (This was caught by
getentropy(3) ATF tests.)
Reported by: kib
Reviewed by: kib
Discussed with: delphij
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14785
The general idea here is to provide userspace programs with well-defined
sources of entropy, in a fashion that doesn't require opening a new file
descriptor (ulimits) or accessing paths (/dev/urandom may be restricted
by chroot or capsicum).
getrandom(2) is the more general API, and comes from the Linux world.
Since our urandom and random devices are identical, the GRND_RANDOM flag
is ignored.
getentropy(3) is added as a compatibility shim for the OpenBSD API.
truss(1) support is included.
Tests for both system calls are provided. Coverage is believed to be at
least as comprehensive as LTP getrandom(2) test coverage. Additionally,
instructions for running the LTP tests directly against FreeBSD are provided
in the "Test Plan" section of the Differential revision linked below. (They
pass, of course.)
PR: 194204
Reported by: David CARLIER <david.carlier AT hardenedbsd.org>
Discussed with: cperciva, delphij, jhb, markj
Relnotes: maybe
Differential Revision: https://reviews.freebsd.org/D14500
In contrast to the existing NetBSD setcontext_link test, these tests
verify that passing from 1 to 6 arguments through to the callback function
work correctly which can be useful for testing ABIs which split arguments
between registers and the stack.
Sponsored by: DARPA / AFRL
libregex is a regex(3) implementation intended to feature GNU extensions and
any other non-POSIX compliant extensions that are deemed worthy.
These extensions are separated out into a separate library for the sake of
not cluttering up libc further with them as well as not deteriorating the
speed (or lack thereof) of the libc implementation.
libregex is implemented as a build of the libc implementation with LIBREGEX
defined to distinguish this from a libc build. The reasons for
implementation like this are two-fold:
1.) Maintenance- This reduces the overhead induced by adding yet another
regex implementation to base.
2.) Ease of use- Flipping on GNU extensions will be as simple as linking
against libregex, and POSIX-compliant compilations can be guaranteed with a
REG_POSIX cflag that should be ignored by libc/regex and disables extensions
in libregex. It is also easier to keep REG_POSIX sane and POSIX pure when
implemented in this fashion.
Tests are added for future functionality, but left disconnected for the time
being while other testing is done.
Reviewed by: cem (previous version)
Differential Revision: https://reviews.freebsd.org/D12934
Currently each call to telldir() requires a malloc and adds an entry to a
linked list which must be traversed on future telldir(), seekdir(),
closedir(), and readdir() calls. Applications that call telldir() for every
directory entry incur O(n^2) behavior in readdir() and O(n) in telldir() and
closedir().
This optimization eliminates the malloc() and linked list in most cases by
packing the relevant information into a single long. On 64-bit architectures
msdosfs, NFS, tmpfs, UFS, and ZFS can all use the packed representation. On
32-bit architectures msdosfs, NFS, and UFS can use the packed
representation, but ZFS and tmpfs can only use it for about the first 128
files per directory. Memory savings is about 50 bytes per telldir(3) call.
Speedup for telldir()-heavy directory traversals is about 20-30x for one
million files per directory.
Reviewed by: kib, mav, mckusick
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13385
getmntinfo(3) is designed around a relatively static or slow growing set of
current mounts. It tried to detect a race with somewhat concurrent mount
and re-call getfsstat(2) in that case, looping indefinitely. It also
allocated space for a single extra mount as slop.
In the case where the user has a large number of mounts and is adding them
at a rapid pace, it fell over.
This patch makes two functional changes:
1. Allocate even more slop. Double whatever the last getfsstat(2) returned.
2. Abort and return some known results after looping a few times
(arbitrarily, 3). If the list is constantly changing, we can't guarantee
we return a full result to the user at any point anyways.
While here, add very basic functional tests for getmntinfo(3) to the libc
suite.
PR: 221743
Submitted by: Peter Eriksson <peter AT ifm.liu.se> (earlier version)
Sponsored by: Dell EMC Isilon
abort_handler_s() currently simply calls abort(), though the standard
specifies more: "Writes an implementation-defined message to stderr
which must include the string pointed to by msg and calls abort()."
memset_s() is missing error condition "n > smax", and does not invoke
the constraint handler after filling the buffer: "following errors are
detected at runtime and call the currently installed constraint
handler function after storing ch in every location of the destination
range [dest, dest+destsz) if dest and destsz are themselves valid",
one of the errors is "n > smax" itself.
Submitted by: Yuri Pankov <yuripv@gmx.com>
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D11991
ATF cleanup routines run in separate processes from the tests themselves, so
they can't share global variables.
Also, setdomainname_test needs to be is_exclusive because the test cases
access a global resource.
PR: 219967
Reviewed by: ngie
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D11188
const char *.
This fixes a bogus set of errors from gcc about strdup not being allowed a NULL
argument.
MFC after: 1 week
Sponsored by: Dell EMC Isilon
- Fix -Wmissing-declaration warning by staticizing run_tests.
- Fix -Wsign-compare warnings by casting size_t types to int
for comparisons.
Reindent some of the code in sdump_hostent(..) to accomodate the
overall changes.
MFC after: 3 days
Sponsored by: Dell EMC Isilon
The pattern used prior to this commit was `calloc(1, n * sizeof(type))`;
the pattern that should be used however is `calloc(n, sizeof(type))`.
MFC after: 3 days
Sponsored by: Dell EMC Isilon
The first test triggers the out of bounds read of the 'left' array. It
only fails when realpath.c is compiled with '-fsanitize=address'.
The other test checks for ENOENT when running into an empty
symlink. This matches NetBSD's realpath(3) semantics. Previously,
empty symlinks were treated like ".".
Submitted by: Jan Kokemц╪ller <jan.kokemueller@gmail.com>
PR: 219154
MFC after: 2 weeks
Adapt glob's match() routine to use a greedy algorithm that avoids
exponential runtime in byzantine inputs.
While here, add a testcase for the byzantine input.
Prompted by: https://research.swtch.com/glob
Authored by: Yves Orton <demerphq at gmail.com>
Obtained from: Perl (33252c318625f3c6c89b816ee88481940e3e6f95)
Sponsored by: Dell EMC Isilon
9899:2011 Appendix K 3.7.4.1.
Other needed supporting types, defines and constraint_handler
infrastructure is added as specified in the C11 spec.
Submitted by: Tom Rix <trix@juniper.net>
Sponsored by: Juniper Networks
Discussed with: ed
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D9903
Differential revision: https://reviews.freebsd.org/D10161
The failing test requires the zh_TW.Big5 locale, which is no longer
installed as of r315568.
Add a note/pointer just in case someone considers re-adding it.
Reported by: Jenkins
Sponsored by: Dell EMC Isilon
Add a clock_nanosleep() syscall, as specified by POSIX.
Make nanosleep() a wrapper around it.
Attach the clock_nanosleep test from NetBSD. Adjust it for the
FreeBSD behavior of updating rmtp only when interrupted by a signal.
I believe this to be POSIX-compliant, since POSIX mentions the rmtp
parameter only in the paragraph about EINTR. This is also what
Linux does. (NetBSD updates rmtp unconditionally.)
Copy the whole nanosleep.2 man page from NetBSD because it is complete
and closely resembles the POSIX description. Edit, polish, and reword it
a bit, being sure to keep any relevant text from the FreeBSD page.
Reviewed by: kib, ngie, jilles
MFC after: 3 weeks
Relnotes: yes
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D10020
The %t{d,u} (ptrdiff_t) tests fail for the following reasons:
- ptrdiff_t is by definition int32_t on !LP64 architectures and int64_t on
LP64 architectures.
- intmax_t is by definition fixed to int64_t on all architectures.
- Some of the code in lib/libc/stdio/... is promoting ptrdiff_t to *intmax_t
when parsing/representing the value.
PR: 191674
MFC after: 1 week
Sponsored by: Dell EMC Isilon
The reasoning here was the same as what was done in r313376:
- Gather as many results as possible instead of failing early and
not testing the rest of the cases.
- Simplify logic when checking test inputs vs outputs and printing
test result.
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Don't exclude i386 from LDBL_MANT_DIG == 64; it works properly in
that case.
While here, replace strcmp + atf_tc_fail with ATF_CHECK_MSG for 2
reasons:
- Gather as many results as possible instead of failing early and
not testing the rest of the cases.
- Simplify logic when checking test inputs vs outputs and printing
test result.
Tested on: amd64, i386
MFC after: 1 week
Sponsored by: Dell EMC Isilon
The previous code used to grab definitions from these openssl/openssh,
but this is no longer needed and is no longer correct. libnetbsd
provides all of the needed definitions
libnetbsd is added to CFLAGS automatically via netbsd-tests.test.mk --
hence all of CFLAGS can be cleared
This contains some new testcases in /usr/tests/...:
- .../lib/libc
- .../lib/libthr
- .../lib/msun
- .../sys/kern
Tested on: amd64, i386
MFC after: 1 month
This uses the same fix as r294894 did for the mlock test. The code from
that commit is moved into a common object file which PROGS supports
building first.
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D8689
Back in 2015 when I reimplemented these functions to use an AVL tree, I
was annoyed by the weakness of the typing of these functions. Both tree
nodes and keys are represented by 'void *', meaning that things like the
documentation for these functions are an absolute train wreck.
To make things worse, users of these functions need to cast the return
value of tfind()/tsearch() from 'void *' to 'type_of_key **' in order to
access the key. Technically speaking such casts violate aliasing rules.
I've observed actual breakages as a result of this by enabling features
like LTO.
I've filed a bug report at the Austin Group. Looking at the way the bug
got resolved, they made a pretty good step in the right direction. A new
type 'posix_tnode' has been added to correspond to tree nodes. It is
still defined as 'void' for source-level compatibility, but in the very
far future it could be replaced by a proper structure type containing a
key pointer.
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D8205
it to lib/libc/tests/sys/Makefile [*]
Even though make -VPACKAGE and make -n install seem to do the right thing,
the effects are a bit different, depending on the build host.
MFC after: 1 week
Obtained from: HardenedBSD (af602f0db) [*]
Reported by: Oliver Pinter <oliver.pinter@hardenedbsd.org> [*]
Sponsored by: Dell EMC Isilon
needlessly
This is already being done by bsd.test.mk
The other subdirectory Makefiles were intentionally left alone
MFC after: 1 week
Sponsored by: Dell EMC Isilon
On particular slow networks, it can (on average) take longer to
resolve hosts to IP* addresses. 20 minutes seemed reasonable for
my work network
This will be solved in a more meaningful way (if possible) using
concurrency in the near future
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
- Some of the lib/libc and lib/thr tests fail
- lib/msun/exp_test:exp2_values now passes with clang 3.8.0
The Makefiles in contrib/netbsd-tests were pruned as they have no value
Sponsored by: EMC / Isilon Storage Division
This is the backing feature to implement C++11 thread storage duration
specified by the thread_local keyword. A destructor for given
thread-local object is registered to be executed at the thread
termination time using __cxa_thread_atexit(). Libc calls the
__cxa_thread_calls_dtors() during exit(3), before finalizers and
atexit functions, and libthr calls the function at the thread
termination time, after the stack unwinding and thread-specific key
destruction.
There are several uncertainties in the API which lacks a formal
specification. Among them:
- is it allowed to register destructors during destructing;
we allow, but limiting the nesting level. If too many iterations
detected, a diagnostic is issued to stderr and thread forcibly
terminates for now.
- how to handle destructors which belong to an unloading dso;
for now, we ignore destructor calls for such entries, and
issue a diagnostic. Linux does prevent dso unload until all
threads with destructors from the dso terminated.
It is supposed that the diagnostics allow to detect real-world
applications relying on the above details and possibly adjust
our implementation. Right now the choices were to provide the slim
API (but that rarely stands the practice test).
Tests are added to check generic functionality and to specify some of
the above implementation choices.
Submitted by: Mahdi Mokhtari <mokhi64_gmail.com>
Reviewed by: theraven
Discussed with: dim (detection of -std=c++11 supoort for tests)
Sponsored by: The FreeBSD Foundation (my involvement)
MFC after: 2 weeks
Differential revisions: https://reviews.freebsd.org/D7224,
https://reviews.freebsd.org/D7427
when importing collation support from Dragonfly/Illumos amdmi3@ tested the
collation branch and reported an issue with Russian collation. John Marino fixed
the issue in Dragonfly and I merged it back to FreeBSD.
Now that Illumos is working on merging our fixes they (Lauri Tirkkonen) found
issues with the commit that fixes the russian collation in UTF-8 that resulted
in a crash with strxfrm(3) and the ISO-8859-5 locale (fixed in FreeBSD r302916).
This small test was written to ensure we do not bring back the old issue with
russian collation while fixing the other issue.
(WITH_SYSTEM_COMPILER: Enable by default) and it's prerequisite: r300354,
caused i386 builds to fail when cross-built on an amd64 host.
Reviewed by: bdrewery, delphij, gjb
Approved by: re (gjb)
convname and dst are guaranteed to be non-NULL by iconv_open(3).
src is an array. Remove these tests for NULL pointers.
While I'm here, eliminate a strlcpy with a correct but suspicious-looking
calculation for the third parameter (i.e. not a simple sizeof).
Compare the strings in-place instead of copying.
Found by: bdrewery
Found by: Coverity
CID: 1130050, 1130056
MFC after: 3 days
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D6338
- Use fgetln instead of fgets; localize complexity related to fgetln(3)
inside the loop.
- Skip over blank lines.
- Skip over lines (properly) that start with a "#"
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
h_aliases is a NULL-terminated rather than fixed-length array. nitems() is not
a valid way to determine its end; instead, check for NULL.
Reported by: Coverity
CID: 1346578
Sponsored by: EMC / Isilon Storage Division
