back to the lever before r343030. For 64-bit machines reduce it slightly,
too. Together with r343030 I bumped the limit up to the value we use at
Netflix to serve 100 Gbit/s of sendfile traffic, and it probably isn't a
good default.
Provide a loader tunable to change vnode pager pbufs count. Document it.
In particular, use ifuncs for __getcontextx_size(), also calculate the
size of the extended save area in resolver. Same for __fillcontextx2().
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Back in 1993, the fgetln (then fgetline) interface was changed to not
return a C string. The change was accomplished by ifdefing out the code
that did the termination. Changing the interface would violate our API
stability rules so remove the old implementation.
Sponsored by: DARPA, AFRL
This optimizes out runtime switch and removes yet another cpuid from
libc.
Note that this is the first use of ifunc in i386 libc, so
ifunc-capable toolchain is required for building runnable userspace on
i386, same as on amd64.
Discussed with: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
ci.FreeBSD.org does not have access to a DNS resolver/network (unlike my test
VM), so in order for the test to pass on the host, it needs to avoid the DNS
lookup by using the numeric host address representation.
PR: 235200
Reviewed by: asomers, lwhsu
Approved by: emaste (mentor)
MFC after: 2 weeks
MFC with: r343362, r343365, r343367-r343368, r343461
Differential Revision: https://reviews.freebsd.org/D19026
The return value (`err`) should be checked; not the `errno` value.
PR: 235200
Approved by: emaste (mentor)
Reviewed by: asomers, lwhsu
MFC after: 28 days
MFC with: r343362, r343365, r343367-r343368
Differential Revision: https://reviews.freebsd.org/D18969
This is meant to clarify the fact that the system call will not fail
with -1/EFAULT, as one might expect, when reading the sendfile(2)
manpage today.
While here, pet the mandoc linter, when dealing with the section that
describes valid values for `flags`.
PR: 232210
MFC after: 2 weeks
Approved by: emaste (mentor)
Reviewed by: glebius, 0mp
Differential Revision: https://reviews.freebsd.org/D18949
I should have only changed the format qualifier with the `size_t` value,
`length`, not the other [`off_t`] value, `dest_file_size`.
MFC after: 1 month
MFC with: r343362, r343365, r343367
Approved by: emaste (mentor; implicit)
Reported by: gcc 8.x
gcc 8.x is more pedantic than clang 7.x with format strings and the tests
passed `void*` variables while supplying `%s` (which is technically
incorrect).
Make the affected `void*` variables use `char*` storage instead to address
this issue, as the compiler will upcast the values to `char*`.
MFC after: 1 month
MFC with: r343362
Approved by: emaste (mentor; implicit)
Reviewed by: asomers
Differential Revision: https://reviews.freebsd.org/D18934
These testcases exercise a number of functional requirements for sendfile(2).
The testcases use IPv4 and IPv6 domain sockets with TCP, and were confirmed
functional on UFS and ZFS. UDP address family sockets cannot be used per the
sendfile(2) contract, thus using UDP sockets is outside the scope of
testing the syscall in positive cases. As seen in
`:s_negative_udp_socket_test`, UDP is used to test the sendfile(2) contract
to ensure that EINVAL is returned by sendfile(2).
The testcases added explicitly avoid testing out `SF_SYNC` due to the
complexity of verifying that support. However, this is a good next logical
item to verify.
The `hdtr_positive*` testcases work to a certain degree (the header
testcases pass), but the trailer testcases do not work (it is an expected
failure). In particular, the value received by the mock server doesn't match
the expected value, and instead looks something like the following (using
python array notation):
`trailer[:]message[1:]`
instead of:
`message[:]trailer[:]`
This makes me think there's a buffer overrun issue or problem with the
offset somewhere in the sendfile(2) system call, but I need to do some
other testing first to verify that the code is indeed sane, and my
assumptions/code isn't buggy.
The `sbytes_negative` testcases that check `sbytes` being set to an
invalid value resulting in `EFAULT` fails today as the other change
(which checks `copyout(9)`) has not been committed [1]. Thus, it
should remain an expected failure (see bug 232210 for more details
on this item).
Next steps for testing sendfile(2):
1. Fix the header/trailer testcases so that they pass.
2. Setup if_tap interface and test with it, instead of using "localhost", per
@asomers's suggestion.
3. Handle short recv(2)'s in `server_cat(..)`.
4. Add `SF_SYNC` support.
5. Add some more negative tests outside the scope of the functional contract.
MFC after: 1 month
Reviewed by: asomers
Approved by: emaste (mentor)
PR: 232210
Sponsored by: Netflix, Inc
Differential Revision: https://reviews.freebsd.org/D18625
An integrity check such as a check-hash or a cross-correlation failed.
The integrity error falls between EINVAL that identifies errors in
parameters to a system call and EIO that identifies errors with the
underlying storage media. EINTEGRITY is typically raised by intermediate
kernel layers such as a filesystem or an in-kernel GEOM subsystem when
they detect inconsistencies. Uses include allowing the mount(8) command
to return a different exit value to automate the running of fsck(8)
during a system boot.
These changes make no use of the new error, they just add it. Later
commits will be made for the use of the new error number and it will
be added to additional manual pages as appropriate.
Reviewed by: gnn, dim, brueffer, imp
Discussed with: kib, cem, emaste, ed, jilles
Differential Revision: https://reviews.freebsd.org/D18765
Based on the description in Linux man page.
Reviewed by: markj, ngie (previous version)
Sponsored by: Mellanox Technologies
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D18837
from the local mapping.
Enable the setting by default.
The article behind the change: https://arxiv.org/abs/1901.01161
Reviewed by: markj
Discussed with: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D18764
As it does for recv*(2), MSG_DONTWAIT indicates that the call should
not block, returning EAGAIN instead. Linux and OpenBSD both implement
this, so the change makes porting easier, especially since we do not
return EINVAL or so when unrecognized flags are specified.
Submitted by: Greg V <greg@unrelenting.technology>
Reviewed by: tuexen
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18728
When presented with an arg string like '-l-', getopt_long will successfully
parse out the 'l' short option, then proceed to match '--' against the first
longopts entry as it later does a strncmp with len=0. This latter bit is
arguably another bug in itself, but presumably not a practical issue as all
callers of parse_long_options are already doing the right thing (except this
one pointed out).
An opt string like '-l-' should be considered malformed and throw a bad
argument rather than behaving as if '--' were passed. It cannot possibly do
what the invoker expects, and it's probably the result of a typo (ls -l- a)
rather than any intent.
Reported by: Tony Overfield <toverfield@yahoo.com>
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18616
Add a short description of the function to the appropriate man page and add
reference to it where it makes sense.
Reviewed by: bcr, markj, 0mp
Approved by: markj
Differential Revision: https://reviews.freebsd.org/D18725
Addition of the new errno values requires adding new elements to
sys_errlist array, which is actually ABI-incompatible, since ELF
records the object size. Expand array in advance to 150 elements so
that we have our users to go over the issue only once, at least until
more than 53 new errors are added.
I did not bumped the symbol version, same as it was not done for
previous increases of the array size. Runtime linker only copies as
much data into binary object on copy relocation as the binary'object
specifies. This is not fixable for binaries which access sys_errlist
directly.
While there, correct comment and calculation of the temporary buffer
size for the message printed for unknown error. The on-stack buffer
is used only for the number and delimiter since r108603.
Requested by: mckusick
Reviewed by: mckusick, yuripv
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D18656
Error messages in gai_strerror(3) vary largely among OSs.
For new software we largely replaced the obsoleted EAI_NONAME and
with EAI_NODATA but we never updated the corresponding message to better
match the intended use. We also have references to ai_flags and ai_family
which are not very descriptive for non-developer end users.
Bring new new error messages based on informational RFC 3493, which has
obsoleted RFC 2553, and make them consistent among the header adn
manpage.
MFC after: 1 month
Differentical Revision: D18630
This fixes the obscure endless loop seen with case-insensitive
patterns containing characters in 128-255 range; originally
found running GNU grep test suite.
Our regex implementation being kludgy translates the characters
in case-insensitive pattern to bracket expression containing both
cases for the character and doesn't correctly handle the case when
original character is in bitmap and the other case is not, falling
into the endless loop going through in p_bracket(), ordinary(),
and bothcases().
Reducing the bitmap to 0-127 range for multibyte locales solves this
as none of these characters have other case mapping outside of bitmap.
We are also safe in the case when the original character outside of
bitmap has other case mapping in the bitmap (there are several of those
in our current ctype maps having unidirectional mapping into bitmap).
Reviewed by: bapt, kevans, pfg
Differential revision: https://reviews.freebsd.org/D18302
There is no reason for it to behave differently from openat(fd, NULL).
Also the handling did not worked because the substituted path was from
the system address space, causing EFAULT.
Submitted by: Jack Halford <jack@gandi.net>
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D18501
The list of syscalls that modify st_atim, st_mtim, and st_ctim was quite out
of date and probably not accurate to begin with. Update it, and make it
clear that the list is open-ended.
Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18410
No references to any of these exist in the tree. The list was also
erratic with different architectures exporting different things
(arm64 and riscv exported none).
Reviewed by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D18425
An issue remains with BIND_NOW and processes using threads. For now,
restore libc's BIND_NOW disable, and also disable BIND_NOW in rtld and
libthr.
A patch is in review (D18400) that likely fixes this issue, but just
disable BIND_NOW pending further testing after it is committed.
PR: 233333
Sponsored by: The FreeBSD Foundation
in threaded programs that unload libraries.
Summary:
The GNOME update to 3.28 exposed a bug in __elf_phdr_match_addr(), which leads
to a crash when building devel/libsoup on powerpc64.
Due to __elf_phdr_match_addr() limiting its search to PF_X sections, on the
PPC64 ELFv1 ABI, it was never matching function pointers properly.
This meant that libthr was never cleaning up its atfork list in
__pthread_cxa_finalize(), so if a library with an atfork handler was unloaded,
libthr would crash on the next fork.
Normally, the null pointer check it does before calling the handler would avoid
this crash, but, due to PPC64 ELFv1 using function descriptors instead of raw
function pointers, a null check against the pointer itself is insufficient, as
the pointer itself was not null, it was just pointing at a function descriptor
that had been zeroed. (Which is an ABI violation.)
Calling a zeroed function descriptor on PPC64 ELFv1 causes a jump to address 0
with a zeroed r2 and r11.
Submitted by: git_bdragon.rtk0.net
Reviewed By: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18364
See the review for sample test results.
Reviewed by: kib (kernel part)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18401
Handling sizes of > 32 backwards will be updated later.
Reviewed by: kib (kernel part)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18387
For non-ERMS case the code used handle possible trailing bytes with
movsb first and then followed it up with movsq. This also happened
to alter how calculations were done for other cases.
Handle the tail with regular movs, just like when copying forward.
Use leaq to calculate the right offset from the get go, instead of
doing separate add and sub.
This adjusts the offset for non-rep cases so that they can be used
to handle the tail.
The routine is still a work in progress.
Sponsored by: The FreeBSD Foundation
When immediate bind mode is requested, as of r340675 rtld processes
irelocs in PLT immediately after other PLT relocs. That addresses the
libc + BIND_NOW startup crash the workaround is no longer needed.
PR: 233333
Matcher function incorrectly assumed that moffset that we get from
findmust is in bytes. Fix this by introducing a stepback function,
taking short path if MB_CUR_MAX is 1, and going back byte-by-byte,
checking if we have a legal character sequence otherwise.
PR: 153502
Reviewed by: pfg, kevans
Approved by: kib (mentor, implicit)
Differential revision: https://reviews.freebsd.org/D18297
That avoids a syscall - getpagesize(3) gets the value from the ELF
aux strings.
Reviewed by: kib
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17989
Building libc WITH_BIND_NOW results in segfault at process start. For
now force BIND_NOW off until the root cause can be identified and fixed.
PR: 233333
Sponsored by: The FreeBSD Foundation
Instead of jumping to locations which store the exact number of bytes,
use displacement to move the destination.
In particular the following clears an area between 8-16 (inclusive)
branch-free:
movq %r10,(%rdi)
movq %r10,-8(%rdi,%rcx)
For instance for rcx of 10 the second line is rdi + 10 - 8 = rdi + 2.
Writing 8 bytes starting at that offset overlaps with 6 bytes written
previously and writes 2 new, giving 10 in total.
Provides a nice win for smaller stores. Other ones are erratic depending
on the microarchitecture.
General idea taken from NetBSD (restricted use of the trick) and bionic
string functions (use for various ranges like in this patch).
Reviewed by: kib (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17660
- tidy up memset to have rax set earlier for small sizes
- finish the tail in memset with an overlapping store
- align memset buffers to 16 bytes before using rep stos
Sponsored by: The FreeBSD Foundation
