screen bitmap and within a single MEMBUF were broken when first source
line is before the first destination line and the sub-bitmaps overlap.
The fix just copies horizontal lines in reverse order when the first
source line is before the first destination line. This switches
directions unnecessarily in some cases, but the switch is about as
fast as doing a precise detection of overlaps. When the first lines
are the same, there can be undetected overlap in the horizontal
direction. The old code already handles this mostly accidentally by
using bcopy() for MEMBUFs and by copying through a temporary buffer
for the screen bitmap although the latter is sub-optimal in direct
modes.
It is a useful arc4random wrapper in the kernel for much the same reasons as
in userspace. Move the source to libkern (because kernel build is
restricted to sys/, but userspace can include any file it likes) and build
kernel and libc versions from the same source file.
Copy the documentation from arc4random_uniform(3) to the section 9 page.
While here, add missing arc4random_buf(9) symlink.
Sponsored by: Dell EMC Isilon
compat mode or not. This is useful when implementing compatibility ioctl(2)
handlers in userspace.
MFC after: 1 week
Sponsored by: Mellanox Technologies
Add fileargs_lstat function to cap_fileargs casper service to be able to
lstat files while in capability mode. It can only lstat files given in
fileargs_init.
Submitted by: Bora Özarslan <borako.ozarslan@gmail.com>
Reviewed by: oshogbo, cem (partial)
MFC after: 3 weeks
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19548
with old kernels, by breaking the support for large frame buffers in the
same way as for current kernels.
Large frame buffers may be too large to map into kva, and the kernel
(syscons) only uses the first screen page anyway, so r203535, r205557
and 248799 limit the buffer size in VESA modes to the first screen
page, apparently without noticing that this breaks applications by
using the same limit for user mappings as for kernel mappings. In
vgl, this makes the virtual screen the same as the physical screen.
However, this is almost a feature since clearing and switching large
(usually mostly unused) frame buffers takes too long. E.g., on a 16
year old low-end AGP card it takes about 12 seconds to clear the 128MB
frame buffer in old kernels that map it all and also map it with slow
attributes (e.g., uncacheable). Older PCI cards are even slower, but
usually have less memory. Newer PCIe cards are faster, but may have
many GB of memory. Also, vgl malloc()s a shadow buffer with the same
size as the frame buffer, so large frame buffers are even more wasteful
in applications than in the kernel.
Use the same limit in vgl as in newer kernels.
Virtual screens and panning still work in non-VESA modes that have
more than 1 page. The reduced buffer size in the kernel also breaks
mmap() of the last physical page in modes where the reduced size is
not a multiple of the physical page size. The same reduction in vgl
only reduces the virtual screen size.
random.3 is only "better" in contrast to rand.3. Both are non-cryptographic
pseudo-random number generators. The opening blurbs of each's DESCRIPTION
section does emphasize this, and correctly directs unfamiliar developers to
arc4random(3). However, the summary (".Nd" or Name description) of random.3
conflicted in tone and message with that warning.
Resolve the conflict by clarifying in the Nd section that random(3) is
non-cryptographic and pseudo-random. Elide the "better" qualifier which
implied a comparison but did not provide a specific object to contrast.
Sponsored by: Dell EMC Isilon
Since inits for the main binary are run from rtld (for some time), the
rtld_exit atexit(3) handler, which is passed from rtld to the program
entry and installed by csu, is installed after any atexit(3) handlers
installed by main binary constructors. This means that rtld_exit() is
fired before main binary handlers.
Typical C++ static constructors are executed from init (either binary
or libs) but use atexit(3) to ensure that destructors are called in
the right order, independent of the linking order. Also, C++
libraries finalizers call __cxa_finalize(3) to flush library'
atexit(3) entries. Since atexit(3) entry is cleared after being run,
this would be mostly innocent, except that, atexit(rtld_exit) done
after main binary constructors, makes destructors from libraries
executed before destructors for main.
Fix by reordering atexit(rtld_exit) before inits for main binary, same
as it happened when inits were called by csu. Do it using new private
libc symbol with pre-defined ABI.
Reported. tested, and reviewed by: kan
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
depths > 8. Add some smaller optimizations for these depths. Use a
more generic method for all depths >= 8, although this gives tiny
pessimizations for these depths.
For clearing the whole frame buffer, avoid the same pessimizations
for depths > 8. Add some larger optimizations for these depths. Use
an even more generic method for all depths >= 8 to give the optimizations
for depths > 8 and a tiny pessimization for depth 8.
The main pessimization was that old versions of bcopy() copy 1 byte at a
time for all trailing bytes. (i386 still does this. amd64 now pessimizzes
large sizes instead of small ones if the CPU supports ERMS. dev/fb gets
this wrong by mostly not using the bcopy() family or the technically correct
bus space functions but by mostly copying 2 bytes at a time using an
unoptimized loop without even volatile declarations to prevent the compiler
rewriting it.)
The sizes here are 1, 2, 3 or 4 bytes, so depths 9-16 were up to twice as
slow as necessary and depths 17-24 were up to 3 times slower than necessary.
Fix this (except depths 17-24 are still up to 2 times slower than necessary)
by using (builtin) memcpy() instead of bcopy() and reorganizing so that the
complier can see the small constant sizes. Reduce special cases while
reorganizing although this is slightly slower than adding special cases.
The compiler inlining (and even -O2 vs -O0) makes little difference compared
with reducing the number of accesses except on modern hardware it gives a
small improvement.
Clearing was also pessimized mainly by the extra accesses. Fix it quite
differently by creating a MEMBUF containing 1 line (in fast memory using
a slow method) and copying this. This is only slightly slower than reducing
everything to efficient memset()s and bcopy()s, but simpler, especially
for the segmented case. This works for planar modes too, but don't use it
then since the old method was actually optimal for planar modes (it works
by moving the slow i/o instructions out of inner loops), while for direct
modes the slow instructions were all in the invisible inner loop in bcopy().
Use htole32() and le32toh() and some type puns instead of unoptimized
functions for converting colors. This optimization is mostly in the noise.
libvgl is only supported on x86, so it could hard-code the assumption that
the byte order is le32, but the old conversion functions didn't hard-code
this.
* Use `MIN` instead of similar hand rolled macro.
* Sort headers.
* Use `errno.h` instead of `sys/errno.h`.
* Wrap the argument to sizeof in parentheses for clarity.
* Remove `__BSD_VISIBLE` and `_XOPEN_SOURCE` #defines to mute warnings about
incompatible snprintf definitions.
This fixes a number of warnings I've been seeing lately in my builds.
Sort makefile variables per style.Makefile(9) (`CFLAGS`/`CWARNFLAG.gcc`) and
bump `WARNS` to 3.
MFC after: 2 weeks
Reviewed by: jtl
Approved by: jtl (mentor)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D19851
Our home-rolled solution didn't quite capture all of the details, and we
didn't actually validate snapshot names at all. zfs_name_valid captures the
important details, but it doesn't necessarily expose the errors that we're
wanting to see in the be_validate_* functions. Validating lengths
independently, then the names, should make this a non-issue.
the same code as the VIDBUF8 case, so it only worked for depths <= 8.
The 2 directions for copying between VIDBUFs and MEMBUFs worked by using
a Read/Write organization which makes the destination a VIDBUF so the
MEMBUF case was not reached, and the VIDBUF cases have already been fixed.
Fix this by removing "optimizations" for the VIDBUF8 case so that the
MEMBUF case can fall through to the general (non-segmented) case. The
optimizations were to duplicate code for the VIDBUF8 case so as to
avoid 2 multiplications by 1 at runtime. This optimization is not useful
since the multiplications are not in the inner loop.
Remove the same "optimization" for the VIDBUF8S case. It was even less
useful there since it duplicated more to do relatively less.
the file associated with the given file descriptor.
Reviewed by: kib, asomers
Reviewed by: cem, jilles, brooks (they reviewed previous version)
Discussed with: pjd, and many others
Differential Revision: https://reviews.freebsd.org/D14567
There are a few places that use hand crafted versions of the macros
from sys/netinet/in.h making it difficult to actually alter the
values in use by these macros. Correct that by replacing handcrafted
code with proper macro usage.
Reviewed by: karels, kristof
Approved by: bde (mentor)
MFC after: 3 weeks
Sponsored by: John Gilmore
Differential Revision: https://reviews.freebsd.org/D19317
provider grows, GELI will expand automatically and will move the metadata
to the new location of the last sector.
This functionality is turned on by default. It can be turned off with the
-R flag, but it is not recommended - if the underlying provider grows and
automatic expansion is turned off, it won't be possible to attach this
provider again, as the metadata is no longer located in the last sector.
If the automatic expansion is turned off and the underlying provider grows,
GELI will only log a message with the previous size of the provider, so
recovery can be easier.
Obtained from: Fudo Security
from 1.0.0:
Add "continuation" flag, to allow multiple "xo" invocations in a single line of output (#58)
Add --top-wrap to make top-level JSON wrappers
Add --{open,close}-{list,instace} options
Add xo_xml_leader(), to detect use of some bogus XML tags. It's still bad form, but it's a little safer now
Avoid call to xo_write before xo_flush, since the latter calls the former
Check return code from xo_flush_h properly (<0) (FreeBSD Bug 236935)
For JSON output, avoid newline before a container's close brace (#62)
Merge branch 'text_only' of https://github.com/zvr/libxo into zvr-text_only
Use XO_USE_INT_RETURN_CODES, not USE_INT_RETURN_CODES
add docs for --continuation
add docs for --not-first
call xo_state_set_flags before values and close containers; add XOIF_MADE_OUTPUT flag to track state; make proper empty JSON objects in xo_finish
color_map code has to be #ifdef'd out, since the struct definition
correct xo_flush_func_t (doesn't use xo_ssize_t)
make depth change for --top-wrap only for JSON
fix to handle --top-wrap in "xo" by being more consistent with handling trailing newlines
fix to handle text-only version #64 (from zvr)
fix xo_buf_has_room for round up to the next XO_BUFSIZ, not just add XO_BUFSIZ to the size (FreeBSD Bug 236937)
update docs for new "xo" options
update functions to use xo_ssize_t
update test cases
from 1.0.1:
Add EINTEGRITY to .pot files under test/gettext/ (fix from FreeBSD)
from 1.0.2:
handle failure from xo_vnsprintf; don't add -1 to "rc"
PR: 236937, 236935
Submitted by: phil
Reported by: Alfonso S. Siciliano <alfix86@gmail.com>
MFC after: 2 weeks
To use bectl in an example, when one creates a new boot environment with
either `bectl create <be>` or `bectl create -e <otherbe> <be>`, libbe will
take a snapshot of the original boot environment to clone. Previously, this
used %F-%T date format as the snapshot name, but this has some limitations-
attempting to create multiple boot environments in quick succession may
collide if done within the same second.
Tack a serial onto it to reduce the chances of a collision... we could still
collide if multiple processes/threads are creating boot environments at the
same time, but this is likely not a big concern as this has only been
reported as occurring in freebsd-ci setup.
MFC after: 3 days
The current approach of injecting manifest into mac_veriexec is to
verify the integrity of it in userspace (veriexec (8)) and pass its
entries into kernel using a char device (/dev/veriexec).
This requires verifying root partition integrity in loader,
for example by using memory disk and checking its hash.
Otherwise if rootfs is compromised an attacker could inject their own data.
This patch introduces an option to parse manifest in kernel based on envs.
The loader sets manifest path and digest.
EVENTHANDLER is used to launch the module right after the rootfs is mounted.
It has to be done this way, since one might want to verify integrity of the init file.
This means that manifest is required to be present on the root partition.
Note that the envs have to be set right before boot to make sure that no one can spoof them.
Submitted by: Kornel Duleba <mindal@semihalf.com>
Reviewed by: sjg
Obtained from: Semihalf
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D19281
This patch adds a new table begemotSnmpdTransInetTable that uses the
InetAddressType textual convention and can be used to create listening
ports for IPv4, IPv6, zoned IPv6 and based on DNS names. It also supports
future extension beyond UDP by adding a protocol identifier to the table
index. In order to support this gensnmptree had to be modified.
Submitted by: harti
MFC after: 1 month
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D16654
Per the upstream pull-request [1]:
```
gtest prior to this change would completely ignore `GTEST_SKIP()` if
called in `Environment::SetUp()`, instead of bailing out early, unlike
`Test::SetUp()`, which would cause the tests themselves to be skipped.
The only way (prior to this change) to skip the tests would be to
trigger a fatal error via `GTEST_FAIL()`.
Desirable behavior, in this case, when dealing with
`Environment::SetUp()` is to check for prerequisites on a system
(example, kernel supports a particular featureset, e.g., capsicum), and
skip the tests. The alternatives prior to this change would be
undesirable:
- Failing sends the wrong message to the test user, as the result of the
tests is indeterminate, not failed.
- Having to add per-test class abstractions that override `SetUp()` to
test for the capsicum feature set, then skip all of the tests in their
respective SetUp fixtures, would be a lot of human and computational
work; checking for the feature would need to be done for all of the
tests, instead of once for all of the tests.
For those reasons, making `Environment::SetUp()` handle `GTEST_SKIP()`,
by not executing the testcases, is the most desirable solution.
In order to properly diagnose what happened when running the tests if
they are skipped, print out the diagnostics in an ad hoc manner.
Update the documentation to note this change and integrate a new test,
gtest_skip_in_environment_setup_test, into the test suite.
This change addresses #2189.
Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
```
The goal with my merging in this change is to avoid requiring extensive
refactoring/retesting of test suites when ensuring prerequisites are met,
e.g., checking for a CAPABILITIES-enabled kernel before running capsicum-test
(see D19758 for more details).
The proof-of-concept is being imported before accepted by the upstream
project due to the fact that the upstream project is undergoing a potential
development freeze and the maintainers aren't responding to my PR.
1. https://github.com/google/googletest/pull/2203
Reported by: asomers (https://github.com/google/googletest/issues/2189)
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D19765
'be_destroy' can destroy a boot environment (by name) or a given snapshot.
If the target to be destroyed is a dataset, check if it's mounted. We don't
want to check if the origin dataset is mounted when destroying a snapshot.
PR: 236043
Submitted by: Rob Fairbanks <rob.fx907 gmail com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19650
The current logic for CSTD/CXXSTD requires homogenity as far as the
supported C/C++ standards, which is a sensible default. However, when
dealing with differing versions of C++, some code may compile with C++11, but
not C++17 (for instance). So in order to avoid having people convert over their
code to the new standard, give the users the ability to specify the standard on
a per-program basis.
This will allow a user to override the supporting standard for a set of
programs, mixing C++11 with C++14 (for instance).
Reviewed by: asomers
Apprved by: emaste (mentor)
MFC after: 1 month
MFC with: r345708
Differential Revision: https://reviews.freebsd.org/D19738
CXXSTD was added as the C++ analogue to CSTD.
CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.
This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.
As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.
Notes:
This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.
Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:
Before this commit:
```
CXXFLAGS+= -std=c++14
```
After this commit:
```
CXXSTD= c++14
```
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 month
MFC with: r345203, r345704, r345705
Relnotes: yes
Tested with: make tinderbox
Differential Revision: https://reviews.freebsd.org/D19732
When a review is closed via Phabricator it updates the patch attached to the
review. I downloaded the raw patch from Phabricator, applied it, and repeated
my mistake from r345704 by accident mixing content from D19732 and D19738.
For my own personal sanity, I will try not to mix reviews like this in the
future.
MFC after: 1 month
MFC with: r345706
Approved by: emaste (mentor, implicit)
CXXSTD was added as the C++ analogue to CSTD.
CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.
This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.
As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.
Notes:
This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.
Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:
Before this commit:
```
CXXFLAGS+= -std=c++14
```
After this commit:
```
CXXSTD= c++14
```
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 month
MFC with: r345203, r345704, r345705
Relnotes: yes
Tested with: make tinderbox
Differential Revision: https://reviews.freebsd.org/D19732
I accidentally committed code from two reviews. I will reintroduce the code to
bsd.progs.mk as part of a separate commit from r345704.
Approved by: emaste (mentor, implicit)
MFC after: 2 months
MFC with: r345704
CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.
This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.
As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.
Notes:
This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.
Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:
Before this commit:
```
CXXFLAGS+= -std=c++14
```
After this commit:
```
CXXSTD= c++14
```
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 month
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D19732
If dso uses initial exec TLS mode, rtld tries to allocate TLS in
static space. If there is no space left, the dlopen(3) fails. If space
if allocated, initial content from PT_TLS segment is distributed to
all threads' pcbs, which was missed and caused un-initialized TLS
segment for such dso after dlopen(3).
The mode is auto-detected either due to the relocation used, or if the
DF_STATIC_TLS dynamic flag is set. In the later case, the TLS segment
is tried to allocate earlier, which increases chance of the dlopen(3)
to succeed. LLD was recently fixed to properly emit the flag, ld.bdf
did it always.
Initial test by: dumbbell
Tested by: emaste (amd64), ian (arm)
Tested by: Gerald Aryeetey <aryeeteygerald_rogers.com> (arm64)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D19072
Correct restoring was only attempted for mode 258 (800x600x4 P). (This
was the only useful graphics mode supported in the kernel until 10-15
years ago, and is still the only one explicitly documented in the man
page). The comment says that it is the geometry (subscreen size) that
is restored, but it seems to only be necessary to restore the font
size, with the geometry only needed since it is set by the same ioctl.
The font size was not restored for this mode, but was forced to 16.
For other graphics modes, the font size was clobbered to 0. This
confuses but doesn't crash the kernel (font size 0 gives null text).
This confuses and crashes vidcontrol. The only way to recover was to
use vidcontrol to set the mode to any text mode on the way back to the
original graphics mode.
vidcontrol gets this wrong in the opposite way when backing out of
changes after an error. It restores the font size correctly, but
forces the geometry to the full screen size.
r80270 has the usual wrong fix for unsafe signal handling -- just set
a flag and return to let an event loop check the flag and do safe
handling. This never works for signals like SIGBUS and SIGSEGV that
repeat and works poorly for others unless the application has an event
loop designed to support this.
For these signals, clean up unsafely as before, except for arranging that
nested signals are fatal and forcing a nested signal if the cleanup doesn't
cause one.
method as in /bin/sh.
We still do technically undefined things in the signal handler, but it
is safe in practice to access state that is protected by INTOFF/INTON.
In a recent commit, I sprinkled VGLMouseFrozen++/-- operations in
places that need INTOFF/INTON. This prevented clobbering of pixels
under the mouse, but left mouse signals deferred for too long. It is
necessary to call the signal handler when the count goes to 0. Old
versions did this in the unfreeze function, but didn't block actual
signals, so the signal handler raced itself. The sprinkled operations
reduced the races, but when then worked to block a race they left
signals deferred for too long.
Use INTOFF/INTON to fix complete loss of mouse signals while reading
the mouse status. Clobbering of the state was prevented by SIG_IGN'ing
mouse signals, but that has a high overhead and broke more than it
fixed by losing mouse signals completely. sigprocmask() works to block
signals without losing them completely, but its overhead is also too
high.
libvgl's mouse signal handling is often worse than none. Applications
can't block waiting for a mouse or keyboard or other event, but have
to busy-wait. The SIG_IGN's lost about half of all mouse events while
busy-waiting for mouse events.
It started truncating its color arg to 8 bits using plot() in r229415.
The version in r229415 is also more than 3 times slower in segmented
modes, by doing more syscalls to move the window.
segmented modes.
Also fix some style bugs in the 2 changed lines. libvgl uses a very non-KNF
style with 2-column indentation with no tabs except for regressions.
r345620 by kib@ fixed the rtld issue that caused a crash at startup
during resolution of libc's ifuncs with BIND_NOW.
PR: 233333
Sponsored by: The FreeBSD Foundation
r322369's use of basename(3) was incorrect and worked by accident so
long as the pidfile path was absolute or consisted of a single
component. Fix the basename() usage and add a regression test.
Reported by: 0mp
Reviewed by: cem
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D19728
Reading of single pixels didn't look under the cursor.
Copying of 1x1 bitmaps didn't look under the cursor for either reading
or writing.
Copying of larger bitmaps looked under the cursor for at most the
destination.
Copying of larger bitmaps looked under a garbage cursor (for the Display
bitmap) when the destination is a MEMBUF. The results are not used, so
this only wasted time and flickered the cursor.
Writing of single pixels looked under a garbage cursor for MEMBUF
destinations, as above except this clobbered the current cursor and
didn't update the MEMBUF. Writing of single pixels is not implemented
yet in depths > 8. Otherwise, writing of single pixels worked. It was
the only working case for accessing pixels under the cursor.
Clearing of MEMBUFs wasted time freezing the cursor in the Display bitmap.
The fixes abuse the top bits in the color arg to the cursor freezing
function to control the function. Also clear the top 8 bits so that
applications can't clobber the control bits or create 256 aliases for
every 24-bit pixel value in depth 32.
Races fixed:
Showing and hiding the cursor only tried to avoid races with the mouse
event signal handler for internal operations. There are still many
shorter races from not using volatile or sig_atomic_t for the variable
to control this. This variable also controls freezes, and has more
complicated states than before.
The internal operation of unfreezing the cursor opened a race window
by unsetting the signal/freeze variable before showing the cursor.
depths for the source and target are not supported. The bits for higher
numbered planes (mostly for red) were either not copied or were copied to
lower numbered planes for nearby pixels.
Quick fix for creation of mouse cursor bitmaps in all depths. This fix is
only complete for the default lightwhite cursor with a black frame.
Even the lightwhite and black colors are hard to find. The templates
use 0xff for lightwhite, but that means brightblue in the simplest mode
(Truecolor depth 24). Other modes are even more complicated -- they are
singly or doubly indirect throught palette(s) and changing of the palettes
by applications is supported.
Details:
Replicate the template value for Truecolor modes to fill out the target
depth (and more for depths not a multiple of 8). Do this for every
drawing of the cursor so that it sort of works for mouse cursor bitmaps
set by applications.
Use 0xf for lightwhite in most other modes. Only do this for the
default cursor so that it doesn't affect mouse cursor bitmaps set by
applications. 0xf mostly works because it was originally for CGA
lightwhite and is emulated using 1 or 2 indirections on EGA and VGA.
0x3f (EGA white) and 0xff (VGA black) direct palette indexes mostly
don't work since backwards compatibility inhibits or prevents them
representing lightwhite. But 0x3f (EGA white) must be used for mode
37 (VGA_MODEX) (320x240x8 V) since this mode is closer to EGA than VGA.
was not taken modulo the window size in VGLClear().
Segmented modes also need a kernel fix to almost work. The ioctl to set
the window origin is broken.
These bugs are rarely problems since non-VESA modes only need
segmentation to support multiple pages but libvgl doesn't support
multiple pages and treats these modes as non-segmented, and VESA modes
are usually mapped linearly except on old hardware so they really are
non-segmented.
to match the change in its declaration. Change the declaration back to
"byte color" since setting of the border color is not supported for more
than 256 colors.
VGLBitmapString() and VGLSetBorder() so as to not truncate to 8 bits.
Complete the corresponding fix for VGLGetXY() and VGLPutXY() (parts
of the man page were out of date).
Since the font format is undocumented, it is unclear how non-multiples
of 8 should be padded to bytes in the font file. Use the same
representation as bdf text format (big- endian, with padding in the
lower bits).
There seems to be no alternative to reading each plane independently using
3 slow i/o's per plane (this delivers 8 nearby pixels, but we don't buffer
the results so run 8 times slower than necessary.
All the code for this was there, but it was ifdefed out and replaced by
simpler code that cannot work in planar modes. The ifdefed out code
was correct except it was missing a volatile declaration, so compilers
optimized the multiple dummy reads in it to a single read.
Support for 16-bit and 32-bit Truecolor modes was supposed to be
complete in r70991 of main.c and in nearby revisions for other files, but
it was broken by the overruns in most cases (all cases were the mouse
is enabled, and most cases where bitmaps are used). r70991 also
uninintentionally added support for depths 9-15, 17-23 and 25-31.
Depth 24 was more obviously broken and its support is ifdefed out. In
the other ranges, only depth 15 is common. It was broken by buffer
overruns in all cases.
bitmap.c:
- the static buffer was used even when it was too small (but it was
large enough to often work accidentally in depth 16)
- the size of the dynamically allocated buffer was too small
- the sizing info bitmap->PixelBytes was not inititialzed in the bitmap
constructor. It often ended up as 0 for MEMBUFs, so using it in more
places gave more null pointer accesses. (It is per-bitmap, but since
conversion between bitmaps of different depths is not supported (except
from 4 bits by padding to 8), it would work better if it were global.)
main.c:
- depths were rounded down instead of up to a multiple of 8, so PixelBytes
was 1 too small for depths above 8 except 16, 24 and 32.
- PixelBytes was not initialized for 4-bit planar modes. It isn't really
used for frame buffer accesses in these modes, but needs to be 1 in
MEMBUF images.
mouse.c:
- the mouse cursor buffers were too small.
vgl.h:
- PixelBytes was not initialized in the static bitmap constructor. It
should be initialized to the value for the current mode, but that is
impossible in a static constructor. Initialize it to -1 so as to
fail if it is used without further initialization.
All modes that are supposed to be supported now don't crash in
nontrivial tests, and almost work. Missing uses of PixelBytes now
give in-bounds wrong pointers instead of overruns. Misconversions of
bitmaps give multiple miscolored mouse cursors instead of 1 white one,
and similarly for bitmaps copied through a MEMBUF.
args (neither MAP_PRIVATE nor MAP_SHARED). It was broken in r271635
and/or r271724 by stricter checking. The compatibility code in r271724
doesn't work for my old binaries (actually new binaries with old
libraries).
PR: needed to test the fix for PR 162373
r343532 noted the difference between "hw.realmem" and "hw.physmem", which I
was previously unaware of. I discovered that neither sysctl had a
description visible via `sysctl -d', so I found where they were defined and
added suitable descriptions. While in the file, I went ahead and added
descriptions for all the others which lacked them. I also updated sysctl.3
accordingly
Reviewed by: kib, bcr
MFC after: 1 weeks
Sponsored by: Panasas
Differential Revision: https://reviews.freebsd.org/D19007
This commit backports revisions 00938b2b228f3b70d3d9e51f29a1505bdad43f1e and
59f90a338bce2376b540ee239cf4e269bf6d68ad from googletest's master branch to
our included version of googletest, which is based on 1.8.1. It adds the
GTEST_SKIP feature, which is very useful for a project like FreeBSD where
some tests depend on particular system configurations.
Reviewed by: ngie
Obtained from: github.com/google/googletest
MFC after: 2 months
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/rS345331
This makes it more consistent with other filesystems, which all end in "fs",
and more consistent with its mount helper, which is already named
"mount_fusefs".
Reviewed by: cem, rgrimes
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19649
[ELF] Support --{,no-}allow-shlib-undefined
Summary:
In ld.bfd/gold, --no-allow-shlib-undefined is the default when
linking an executable. This patch implements a check to error on
undefined symbols in a shared object, if all of its DT_NEEDED entries
are seen.
Our approach resembles the one used in gold, achieves a good balance
to be useful but not too smart (ld.bfd traces all DSOs and emulates
the behavior of a dynamic linker to catch more cases).
The error is issued based on the symbol table, different from
undefined reference errors issued for relocations. It is most
effective when there are DSOs that were not linked with -z defs (e.g.
when static sanitizers runtime is used).
gold has a comment that some system libraries on GNU/Linux may have
spurious undefined references and thus system libraries should be
excluded (https://sourceware.org/bugzilla/show_bug.cgi?id=6811). The
story may have changed now but we make --allow-shlib-undefined the
default for now. Its interaction with -shared can be discussed in the
future.
Reviewers: ruiu, grimar, pcc, espindola
Reviewed By: ruiu
Subscribers: joerg, emaste, arichardson, llvm-commits
Differential Revision: https://reviews.llvm.org/D57385
Pull in r352943 from upstream lld trunk (by Fangrui Song):
[ELF] Default to --no-allow-shlib-undefined for executables
Summary:
This follows the ld.bfd/gold behavior.
The error check is useful as it captures a common type of ld.so
undefined symbol errors as link-time errors:
// a.cc => a.so (not linked with -z defs)
void f(); // f is undefined
void g() { f(); }
// b.cc => executable with a DT_NEEDED entry on a.so
void g();
int main() { g(); }
// ld.so errors when g() is executed (lazy binding) or when the program is started (-z now)
// symbol lookup error: ... undefined symbol: f
Reviewers: ruiu, grimar, pcc, espindola
Reviewed By: ruiu
Subscribers: llvm-commits, emaste, arichardson
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D57569
Together, these add support for --no-allow-shlib-undefined, and make it
the default for executables, so they will fail to link if any symbols
from needed shared libraries are undefined.
Reported by: jbeich
PR: 236062, 236141
MFC after: 1 month
X-MFC-With: r344779
enabling the functions that save and restore MXCSR, since access to this
register requires SSE support.
Note that you may run into other issues with OpenMP on i386, since this
*not* yet supported upstream, and certainly not extensively tested.
PR: 236062, 236582
MFC after: 1 month
X-MFC-With: r344779
on scalbn and a few other math functions, via libcompiler-rt. This
should allow OpenMP programs to link with BFD linkers too.
Reported by: jbeich
PR: 236062, 236581
MFC after: 1 month
X-MFC-With: r344779
functionality. This should make example OpenMP programs work out of the
box.
Reported by: jbeich
PR: 236062, 236581
MFC after: 1 month
X-MFC-With: r344779
* Set MK_OPENMP to yes by default only on amd64, for now.
* Bump __FreeBSD_version to signal this addition.
* Ensure gcc's conflicting omp.h is not installed if MK_OPENMP is yes.
* Update OptionalObsoleteFiles.inc to cope with the conflicting omp.h.
* Regenerate src.conf(5) with new WITH/WITHOUT fragments.
Relnotes: yes
PR: 236062
MFC after: 1 month
X-MFC-With: r344779
Not connected to the main build yet, as there is still the issue of the
GNU omp.h header conflicting with the LLVM one. (That is, if MK_GCC is
enabled.)
PR: 236062
MFC after: 1 month
X-MFC-With: r344779
This initial integration takes googlemock/googletest release 1.8.1, integrates
the library, tests, and sample unit tests into the build.
googlemock/googletest's inclusion is optionally available via `MK_GOOGLETEST`.
`MK_GOOGLETEST` is dependent on `MK_TESTS` and is enabled by default when
built with a C++11 capable toolchain.
Google tests can be specified via the `GTESTS` variable, which, in comparison
with the other test drivers, is more simplified/streamlined, as Googletest only
supports C++ tests; not raw C or shell tests (C tests can be written in C++
using the standard embedding methods).
No dependent libraries are assumed for the tests. One must specify `gmock`,
`gmock_main`, `gtest`, or `gtest_main`, via `LIBADD` for the program.
More information about googlemock and googletest can be found on the
Googletest [project page](https://github.com/google/googletest), and the
[GoogleMock](https://github.com/google/googletest/blob/v1.8.x/googlemock/docs/Documentation.md)
and
[GoogleTest](https://github.com/google/googletest/tree/v1.8.x/googletest/docs)
docs.
These tests are originally integrated into the build as plain driver tests, but
will be natively integrated into Kyua in a later version.
Known issues/Errata:
* [WhenDynamicCastToTest.AmbiguousCast fails on FreeBSD](https://github.com/google/googletest/issues/2172)
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D19551
Move LLVM's libunwind to its own contrib/ directory similar to other
runtime libraries like libc++ and libcxxrt.
Reviewed by: dim, emaste
Differential Revision: https://reviews.freebsd.org/D19534
This is not required of a compliant implementation, but it's easy to
check for and helps improve compatibility with other common
implementations. Moreover, it's consistent with our
pthread_mutex_destroy().
PR: 234805
Reviewed by: jhb, kib, ngie
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19496
UEFI related headers were copied from edk2.
A new build option "MK_LOADER_EFI_SECUREBOOT" was added to allow
loading of trusted anchors from UEFI.
Certificate revocation support is also introduced.
The forbidden certificates are loaded from dbx variable.
Verification fails in two cases:
There is a direct match between cert in dbx and the one in the chain.
The CA used to sign the chain is found in dbx.
One can also insert a hash of TBS section of a certificate into dbx.
In this case verifications fails only if a direct match with a
certificate in chain is found.
Submitted by: Kornel Duleba <mindal@semihalf.com>
Reviewed by: sjg
Obtained from: Semihalf
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D19093
clang and various other executables will fail to link with undefined
symbols.
Reported by: O. Hartmann <ohartmann@walstatt.org>
MFC after: 1 month
X-MFC-With: r344779
The armv6 build failed in CI due to missing symbols (from these two
source files) in the bootstrap Clang.
This affected only armv6 because other Clang-using archs are using LLD
as the bootstrap linker, and thus include SRCS_MIW via LLD_BOOTSTRAP.
Reported by: CI, via lwhsu
Sponsored by: The FreeBSD Foundation
Use SOURCE_DATE_EPOCH for BUILD_UTC if MK_REPRODUCIBLE_BUILD is yes.
Default SOURCE_DATE_EPOCH to 2019-01-01
Reviewed by: emaste
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D19464
These are taken directly from the density report from a TS1160
tape drive. (Using mt getdensity)
A TS1160 drive stores 20TB raw (60TB with compression) on a JE tape.
lib/libmt/mtlib.c:
Add 3592A6 encrypted/unencrypted density codes, and bpmm/bpi
values.
usr.bin/mt/mt.1:
Add 3592B5 encrypted/unencrypted density codes, bpmm/bpi
values and number of tracks. Bump the man page date.
MFC after: 3 days
Sponsored by: Spectra Logic
According to 0mp, macros are not expanded in the argument provided to
-width. Use plain identifiers for width specification.
Noted and reviewed by: 0mp
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D19308
Instead of PRIVATELIB + NO_PIC. This avoids the need for the wlandebug
PIE special case added in r344211, and provides a stronger guarantee
against 3rd party software coming to depend on the API or ABI.
If / when we declare the API/ABI to be stable we can make it a normal
library.
Discussed with: bapt
Sponsored by: The FreeBSD Foundation
The move to /usr/include/private prefixed paths seems to require a bit more
effort in order to compile programs.
Install the headers to /usr/include/private/g{mock,test}/... and automatically
include /usr/include/private in GTESTS_CXXFLAGS to make compilation seamless. I
will work on the more global problem later with @bdrewery.
Long story short, some of the tests were failing because they expected either
dynamic_cast or RTTI to be functional and it wasn't.
Move all common CXXFLAGS out to googletest.test.inc.mk and reference it from
googletest.test.mk and .../googletest/Makefile.inc
The key difference is that some of the programs were previously being compiled
and installed as tests, which is incorrect. Treat them like helpers instead.
My previous work to integrate these tests was incomplete/incorrect, because I
misunderstood how the cmake macros worked.
This addresses items with the gtest tests, which in turn fixes test compilation
and adds more tests which I had previously missed.
Due to an unknown issue with gtest_stress_test, I had to add pthread to LIBADD,
even though I shouldn't have added it to that (it was failing to link -lpthread
to libprivategtest.a). Add a XXX comment to note that something's awry there
and deserves additional investigation.
The former is from googletest.test.mk, whereas the latter is from plain.test.mk.
As noted in r344328, Kyua will adopt more native GoogleTest support. Thus, it's
more desirable to make the test interface more of an opaque blackbox for the
testcase implementer.
- Look up the corresponding non-*_main libraries via LDFLAGS using the
directories provided in src.libnames.mk. This will allow the libraries to
be built in the "make libraries" phase of buildworld.
- gtest_main relies on gtest, but didn't explicitly call out the dependency
in `LIBADD`. Fill in this missing blank.
They are supporting libraries and as such, will need to be built/installed
when MK_TESTS == no during buildworld, i.e., the "make libraries" phase.
Otherwise, dependent components cannot rely on the libraries, like
`cddl/usr.sbin/zfsd/tests`.
These libraries don't compile on non-C++-11 capable compilers, e.g., g++ 4.2.1
and its corresponding implementation of the c++ library, i.e., libstdc++.
Blacklist compilation on all non-C++-11 capable compilers and give others the
option of opting out of building/installing gmock/gtest via MK_GOOGLETEST.
This option is controlled by MK_CXX and MK_TESTS, as ATF compilation is.
In short, the prior code was far too simplistic when it came to calling recv(2)
and failed intermittently (or in the case of Jenkins, deterministically).
Handle short recv(2)s by checking the return code and incrementing the window
into the buffer by the number of received bytes. If the number of received
bytes <= 0, then bail out of the loop, and test the total number of received
bytes vs the expected number of bytes sent for equality, and base whether or
not the test passes/fails on that fact.
Remove the expected failure, now that the hdtr testcases deterministically pass
on my host after this change [1].
PR: 234809 [1], 235200
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19188
This was discovered through examination -- acl_copy_entry() copies the
tag type and permset fields.
Reviewed by: trasz, pfg
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D19240
reports snap counts of how much a zone alloced and how much it freed. It
may happen that snap values doesn't match, e.g alloced - freed < 0.
Workaround that in memstat library.
Reported by: pho
back to the lever before r343030. For 64-bit machines reduce it slightly,
too. Together with r343030 I bumped the limit up to the value we use at
Netflix to serve 100 Gbit/s of sendfile traffic, and it probably isn't a
good default.
Provide a loader tunable to change vnode pager pbufs count. Document it.
Building binaries as PIE allows the executable itself to be loaded at a
random address when ASLR is enabled (not just its shared libraries).
With this change PIE objects have a .pieo extension and INTERNALLIB
libraries libXXX_pie.a.
MK_PIE is disabled for some kerberos5 tools, Clang, and Subversion, as
they explicitly reference .a libraries in their Makefiles. These can
be addressed on an individual basis later. MK_PIE is also disabled for
rtld-elf because it is already position-independent using bespoke
Makefile rules.
Currently only dynamically linked binaries will be built as PIE.
Discussed with: dim
Reviewed by: kib
MFC after: 1 month
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18423
In particular, use ifuncs for __getcontextx_size(), also calculate the
size of the extended save area in resolver. Same for __fillcontextx2().
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
be_destroy is documented to recursively destroy a boot environment. In the
case of snapshots, one would take this to mean that these are also
recursively destroyed. However, this was previously not the case.
be_destroy would descend into the be_destroy callback and attempt to
zfs_iter_children on the top-level snapshot, which is bogus.
Our alternative approach is to take note of the snapshot name and iterate
through all of fs children of the BE to try destruction in the children.
The -o option is also fixed to work properly with deep BEs. If the BE was
created with `bectl create -e otherDeepBE newDeepBE`, for instance, then a
recursive snapshot of otherDeepBE would have been taken for construction of
newDeepBE but a subsequent destroy with BE_DESTROY_ORIGIN set would only
clean up the snapshot at the root of otherDeepBE: ${BEROOT}/otherDeepBE@...
The most recent iteration instead pretends not to know how these things
work, verifies that the origin is another BE and then passes that back
through be_destroy to DTRT when snapshots and deep BEs may be in play.
MFC after: 1 week
Sync libarchive with vendor.
Relevant vendor changes:
PR #1085: Fix a null pointer dereference bug in zip writer
PR #1110: ZIP reader added support for XZ, LZMA, PPMD8 and BZIP2
decopmpression
PR #1116: Add support for 64-bit ar format
PR #1120: Fix a 7zip crash [1] and a ISO9660 infinite loop [2]
PR #1125: RAR5 reader - fix an invalid read and a memory leak
PR #1131: POSIX reader - do not fail when tree_current_lstat() fails
due to ENOENT [3]
PR #1134: Delete unnecessary null pointer checks before calls of free()
OSS-Fuzz 10843: Force intermediate to uint64_t to make UBSAN happy.
OSS-Fuzz 11011: Avoid buffer overflow in rar5 reader
PR: 233006 [3]
Security: CVE-2019-1000019 [1], CVE-2019-1000020 [2]
MFC after: 2 weeks
nvpair_create_stringv: free the temporary string; this fix affects
nvlist_add_stringf() and nvlist_add_stringv().
nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): free the chain
of nvpairs (as resetting it prevents nvlist_destroy() from freeing it).
Note: freeing the chain in nvlist_destroy() is not sufficient, because
it would still leak through nvlist_take_nvlist_array(). This affects
all nvlist_*_nvlist_array() use
Submitted by: Mindaugas Rasiukevicius <rmind@netbsd.org>
Reported by: clang/gcc ASAN
MFC after: 2 weeks
Currently origin snapshots are left behind when a BE is destroyed, whether
it was an auto-created snapshot or explicitly specified via, for example,
`bectl create -e be@mysnap ...`.
Removing it automatically could be argued as a POLA violation in some
circumstances, so provide a flag to be_destroy for it. An accompanying
option will be added to bectl(8) to utilize this.
Some minor style/consistency nits in the affected areas also addressed.
Reported by: Shawn Webb
MFC after: 1 week
Replace calls to sinf(x) and cosf(x) with a single call to sincosf().
Submitted by: Steve Kargl <sgk@troutmask.apl.washington.edu>
Reviewed by: bde
Approved by: grog
MFC after: 3 days
trig_test.reduction test cases to fail, if the fixes from r343916 have
not yet been applied to the base compiler.
Reported by: lwhsu
PR: 234040
Upstream PR: https://bugs.llvm.org/show_bug.cgi?id=40206
MFC after: 1 week
[X86] Add FPSW as a Def on some FP instructions that were missing it.
Pull in r353141 from upstream llvm trunk (by Craig Topper):
[X86] Connect the default fpsr and dirflag clobbers in inline
assembly to the registers we have defined for them.
Summary:
We don't currently map these constraints to physical register numbers
so they don't make it to the MachineIR representation of inline
assembly.
This could have problems for proper dependency tracking in the
machine schedulers though I don't have a test case that shows that.
Reviewers: rnk
Reviewed By: rnk
Subscribers: eraman, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D57641
Pull in r353489 from upstream llvm trunk (by Craig Topper):
[X86] Add FPCW as a register and start using it as an implicit use on
floating point instructions.
Summary:
FPCW contains the rounding mode control which we manipulate to
implement fp to integer conversion by changing the roudning mode,
storing the value to the stack, and then changing the rounding mode
back. Because we didn't model FPCW and its dependency chain, other
instructions could be scheduled into the middle of the sequence.
This patch introduces the register and adds it as an implciit def of
FLDCW and implicit use of the FP binary arithmetic instructions and
store instructions. There are more instructions that need to be
updated, but this is a good start. I believe this fixes at least the
reduced test case from PR40529.
Reviewers: RKSimon, spatel, rnk, efriedma, andrew.w.kaylor
Subscribers: dim, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D57735
These should fix a problem in clang 7.0 where it would sometimes emit
long double floating point instructions in a slightly wrong order,
leading to failures in our libm tests. In particular, the cbrt_test
test case 'cbrtl_powl' and the trig_test test case 'reduction'.
Also bump __FreeBSD_cc_version, to be able to detect this in our test
suite.
Reported by: lwhsu
PR: 234040
Upstream PR: https://bugs.llvm.org/show_bug.cgi?id=40206
MFC after: 1 week
Back in 1993, the fgetln (then fgetline) interface was changed to not
return a C string. The change was accomplished by ifdefing out the code
that did the termination. Changing the interface would violate our API
stability rules so remove the old implementation.
Sponsored by: DARPA, AFRL
Refactor the function calls and tests so that, on UFS, the proper fields
are filled out.
PR: 233849
Reported by: Andre Albsmeier
Reviewed by: mav, delphij
MFC after: 1 month
Sponsored by: iXsystems Inc
Differential Revision: https://reviews.freebsd.org/D18785
This optimizes out runtime switch and removes yet another cpuid from
libc.
Note that this is the first use of ifunc in i386 libc, so
ifunc-capable toolchain is required for building runnable userspace on
i386, same as on amd64.
Discussed with: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
from its parent so that LOG_PERROR would work. However, this caused
dhclient(8)'s stdio streams to remain open across daemonization, breaking
the ability to capture its foreground output as done in netconfig_ipv4.
Fix this by reverting r341692 and instead passing the parent's stderr
descriptor as an argument to cap_openlog() only when LOG_PERROR is specified
in logopt.
PR: 234514
Suggested by: markj
Reported by: Shawn Webb
Reviewed by: markj, oshogbo
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18989
Use recent best practices for Copyright form at the top of
the license:
1. Remove all the All Rights Reserved clauses on our stuff. Where we
piggybacked others, use a separate line to make things clear.
2. Use "Netflix, Inc." everywhere.
3. Use a single line for the copyright for grep friendliness.
4. Use date ranges in all places for our stuff.
Approved by: Netflix Legal (who gave me the form), adrian@ (pmc files)
When libthr is statically linked into the binary, order of the
constructors execution is not deterministic. It is possible for the
application constructor to use pthread_mutex_* functions before the
libthr initialization was done.
Handle it by:
- making thr_malloc.c locking functions operational when curthread is not
yet set;
- making __thr_malloc_init() idempotent, allowing more than one call to it;
- unconditionally calling __thr_malloc_init() before initializing
a process-private mutex.
Reported and tested by: mmel
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
ci.FreeBSD.org does not have access to a DNS resolver/network (unlike my test
VM), so in order for the test to pass on the host, it needs to avoid the DNS
lookup by using the numeric host address representation.
PR: 235200
Reviewed by: asomers, lwhsu
Approved by: emaste (mentor)
MFC after: 2 weeks
MFC with: r343362, r343365, r343367-r343368, r343461
Differential Revision: https://reviews.freebsd.org/D19026
Effectively all i386 kernels now have two pmaps compiled in: one
managing PAE pagetables, and another non-PAE. The implementation is
selected at cold time depending on the CPU features. The vm_paddr_t is
always 64bit now. As result, nx bit can be used on all capable CPUs.
Option PAE only affects the bus_addr_t: it is still 32bit for non-PAE
configs, for drivers compatibility. Kernel layout, esp. max kernel
address, low memory PDEs and max user address (same as trampoline
start) are now same for PAE and for non-PAE regardless of the type of
page tables used.
Non-PAE kernel (when using PAE pagetables) can handle physical memory
up to 24G now, larger memory requires re-tuning the KVA consumers and
instead the code caps the maximum at 24G. Unfortunately, a lot of
drivers do not use busdma(9) properly so by default even 4G barrier is
not easy. There are two tunables added: hw.above4g_allow and
hw.above24g_allow, the first one is kept enabled for now to evaluate
the status on HEAD, second is only for dev use.
i386 now creates three freelists if there is any memory above 4G, to
allow proper bounce pages allocation. Also, VM_KMEM_SIZE_SCALE changed
from 3 to 1.
The PAE_TABLES kernel config option is retired.
In collaboarion with: pho
Discussed with: emaste
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D18894
The need to use libc malloc(3) from some places in libthr always
caused issues. For instance, per-thread key allocation was switched to
use plain mmap(2) to get storage, because some third party mallocs
used keys for implementation of calloc(3).
Even more important, libthr calls calloc(3) during initialization of
pthread mutexes, and jemalloc uses pthread mutexes. Jemalloc provides
some way to both postpone the initialization, and to make
initialization to use specialized allocator, but this is very fragile
and often breaks. See the referenced PR for another example.
Add the small malloc implementation used by rtld, to libthr. Use it in
thr_spec.c and for mutexes initialization. This avoids the issues with
mutual dependencies between malloc and libthr in principle. The
drawback is that some more allocations are not interceptable for
alternate malloc implementations. There should be not too much memory
use from this allocator, and the alternative, direct use of mmap(2) is
obviously worse.
PR: 235211
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D18988
This includes the bump for cdevsw d_version. Otherwise, the impact on
the ABI (not KBI) is surprisingly low. The most important affected
interface is devname(3) and ttyname(3) which already correctly handle
long names (and ttyname(3) should not be affected at all).
Still, due to the d_version bump, I argue that the change is not MFC-able.
Requested by: mmacy
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D18932
The strdup() call does not take advantage of the known length of the
source string. Replace by malloc() and memcpy() utilizimng the pre-
calculated string length.
Submitted by: cperciva
Reported by: rgrimes
MFC after: 2 weeks
Casper library should not use exit(3) function because before setting it up
applications may register it. Casper doesn't depend on any registered exit
function, so it safe to change this.
Reported by: jceel
MFC after: 2 weeks
The return value (`err`) should be checked; not the `errno` value.
PR: 235200
Approved by: emaste (mentor)
Reviewed by: asomers, lwhsu
MFC after: 28 days
MFC with: r343362, r343365, r343367-r343368
Differential Revision: https://reviews.freebsd.org/D18969
This is meant to clarify the fact that the system call will not fail
with -1/EFAULT, as one might expect, when reading the sendfile(2)
manpage today.
While here, pet the mandoc linter, when dealing with the section that
describes valid values for `flags`.
PR: 232210
MFC after: 2 weeks
Approved by: emaste (mentor)
Reviewed by: glebius, 0mp
Differential Revision: https://reviews.freebsd.org/D18949
I should have only changed the format qualifier with the `size_t` value,
`length`, not the other [`off_t`] value, `dest_file_size`.
MFC after: 1 month
MFC with: r343362, r343365, r343367
Approved by: emaste (mentor; implicit)
Reported by: gcc 8.x
gcc 8.x is more pedantic than clang 7.x with format strings and the tests
passed `void*` variables while supplying `%s` (which is technically
incorrect).
Make the affected `void*` variables use `char*` storage instead to address
this issue, as the compiler will upcast the values to `char*`.
MFC after: 1 month
MFC with: r343362
Approved by: emaste (mentor; implicit)
Reviewed by: asomers
Differential Revision: https://reviews.freebsd.org/D18934
These testcases exercise a number of functional requirements for sendfile(2).
The testcases use IPv4 and IPv6 domain sockets with TCP, and were confirmed
functional on UFS and ZFS. UDP address family sockets cannot be used per the
sendfile(2) contract, thus using UDP sockets is outside the scope of
testing the syscall in positive cases. As seen in
`:s_negative_udp_socket_test`, UDP is used to test the sendfile(2) contract
to ensure that EINVAL is returned by sendfile(2).
The testcases added explicitly avoid testing out `SF_SYNC` due to the
complexity of verifying that support. However, this is a good next logical
item to verify.
The `hdtr_positive*` testcases work to a certain degree (the header
testcases pass), but the trailer testcases do not work (it is an expected
failure). In particular, the value received by the mock server doesn't match
the expected value, and instead looks something like the following (using
python array notation):
`trailer[:]message[1:]`
instead of:
`message[:]trailer[:]`
This makes me think there's a buffer overrun issue or problem with the
offset somewhere in the sendfile(2) system call, but I need to do some
other testing first to verify that the code is indeed sane, and my
assumptions/code isn't buggy.
The `sbytes_negative` testcases that check `sbytes` being set to an
invalid value resulting in `EFAULT` fails today as the other change
(which checks `copyout(9)`) has not been committed [1]. Thus, it
should remain an expected failure (see bug 232210 for more details
on this item).
Next steps for testing sendfile(2):
1. Fix the header/trailer testcases so that they pass.
2. Setup if_tap interface and test with it, instead of using "localhost", per
@asomers's suggestion.
3. Handle short recv(2)'s in `server_cat(..)`.
4. Add `SF_SYNC` support.
5. Add some more negative tests outside the scope of the functional contract.
MFC after: 1 month
Reviewed by: asomers
Approved by: emaste (mentor)
PR: 232210
Sponsored by: Netflix, Inc
Differential Revision: https://reviews.freebsd.org/D18625
Previously, we directly used libzfs_core's lzc_receive to import to a
temporary snapshot, then cloned the snapshot and setup the properties. This
failed when attempting to import replication streams with questionable
error.
libzfs's zfs_receive is a much better fit here, so we now use it instead
with the destination dataset and let libzfs take care of the dirty details.
be_import is greatly simplified as a result.
Reported by: Marie Helene Kvello-Aune <freebsd@mhka.no>
MFC after: 1 week
An integrity check such as a check-hash or a cross-correlation failed.
The integrity error falls between EINVAL that identifies errors in
parameters to a system call and EIO that identifies errors with the
underlying storage media. EINTEGRITY is typically raised by intermediate
kernel layers such as a filesystem or an in-kernel GEOM subsystem when
they detect inconsistencies. Uses include allowing the mount(8) command
to return a different exit value to automate the running of fsck(8)
during a system boot.
These changes make no use of the new error, they just add it. Later
commits will be made for the use of the new error number and it will
be added to additional manual pages as appropriate.
Reviewed by: gnn, dim, brueffer, imp
Discussed with: kib, cem, emaste, ed, jilles
Differential Revision: https://reviews.freebsd.org/D18765
This is CVS revision 1.31 from NetBSD lib/libedit/chartype.c:
Make sure that argv is NULL terminated since functions like tty_stty rely
on it to be so (Gerry Swinslow)
This broke when the wide-character support was enabled in libedit. The
conversion from multibyte to wide-character did not supply the apparently
expected terminating NULL in the new argv array.
PR: 233343
Submitted by: Yuichiro NAITO
Obtained from: NetBSD
MFC after: 1 week
Based on the description in Linux man page.
Reviewed by: markj, ngie (previous version)
Sponsored by: Mellanox Technologies
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D18837
two zones sharing a keg may have different limits. Now this is going
to work:
zone = uma_zcreate();
uma_zone_set_max(zone, limit);
zone2 = uma_zsecond_create(zone);
uma_zone_set_max(zone2, limit2);
Kegs no longer have uk_maxpages field, but zones have uz_items. When
set, it may be rounded up to minimum possible CPU bucket cache size.
For small limits bucket cache can also be reconfigured to be smaller.
Counter uz_items is updated whenever items transition from keg to a
bucket cache or directly to a consumer. If zone has uz_maxitems set and
it is reached, then we are going to sleep.
o Since new limits don't play well with multi-keg zones, remove them. The
idea of multi-keg zones was introduced exactly 10 years ago, and never
have had a practical usage. In discussion with Jeff we came to a wild
agreement that if we ever want to reintroduce the idea of a smart allocator
that would be able to choose between two (or more) totally different
backing stores, that choice should be made one level higher than UMA,
e.g. in malloc(9) or in mget(), or whatever and choice should be controlled
by the caller.
o Sleeping code is improved to account number of sleepers and wake them one
by one, to avoid thundering herd problem.
o Flag UMA_ZONE_NOBUCKETCACHE removed, instead uma_zone_set_maxcache()
KPI added. Having no bucket cache basically means setting maxcache to 0.
o Now with many fields added and many removed (no multi-keg zones!) make
sure that struct uma_zone is perfectly aligned.
Reviewed by: markj, jeff
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D17773
Summary:
GCC expects to link in a crtsavres.o on powerpc platforms. On
powerpc64 this is an empty file, but on powerpc and powerpcspe this does contain
some save/restore functions, which may not actually be necessary for newer
modern GCC and clang. This appeases the in-tree gcc, though, and is needed in
order to switch to the BSD CRTRBEGIN.
PR: 233751
Reviewed By: andrew
Differential Revision: https://reviews.freebsd.org/D18826
Highlights:
- Make sure that only TLS sections are sorted into TLS segment.
- Fixed multiple errors in "Section to Segment mapping".
- Man page updates
- ar improvements
- elfcopy: avoid filter_reloc uninitialized variable for rela
- elfcopy: avoid stripping relocations from static binaries
- readelf: avoid printing directory in front of absolute path
- readelf: add NT_FREEBSD_FEATURE_CTL FreeBSD note type
- test improvements
NOTES:
Some of these changes originated in FreeBSD and simply reduce diffs
between contrib and vendor.
ELF Tool Chain ar is not (currently) used in FreeBSD, and there are
improvements in both FreeBSD and ELF Tool Chain ar that are not in
the other.
Sponsored by: The FreeBSD Foundation
This set of changes is geared towards making bectl respect deep boot
environments when they exist and are mounted. The deep BE composition
functionality (`bectl add`) remains disabled for the time being. This set of
changes has no effect for the average user. but allows deep BE users to
upgrade properly with their current setup.
libbe(3): Open the target boot environment and get a zfs handle, then pass
that with the target mountpoint to be_mount_iter; If the BE_MNT_DEEP flag is
set call zfs_iter_filesystems and mount the child datasets.
Similar logic is employed when unmounting the datasets, save for children
are unmounted first.
bectl(8): Change bectl_cmd_jail to pass the BE_MNT_DEEP flag when
calling be_mount as well as call be_unmount when cleaning up after the
jail has exited instead of umount(2) directly.
PR: 234795
Submitted by: Wes Maag <jwmaag_gmail.com> (test additions by kevans)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18796
We could perhaps have a method that does this given a dataset, but it's yet
clear that we'll always want to bypass the altroot when we grab the
mountpoint. For now, we'll refactor things a bit so we grab the altroot
length when libbe is initialized and have a common method that does the
necessary augmentation (replace with / if it's the root, return a pointer to
later in the string if not).
This will be used in some upcoming work to make be_mount work properly for
deep BEs.
MFC after: 1 week
from the local mapping.
Enable the setting by default.
The article behind the change: https://arxiv.org/abs/1901.01161
Reviewed by: markj
Discussed with: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D18764
j is int32_t and thus j<<31 is undefined if j==1.
Hinted by: muusl-lib (git 688d3da0f1730daddbc954bbc2d27cc96ceee04c)
Discussed with: freebsd-numerics (kargl)
Previously, the following sequence of events was feasible under some
circumstance:
bectl create test
bectl activate test
# the test BE dataset gets promoted and set as bootfs
bectl destroy test
I was unable to reproduce the destroy succeeding, but we should be rejecting
this before it even gets to libzfs because it would leave the system in an
inconsistent state. Forcing the user to be explicit as to which environment
should be activated instead is much better.
Reported by: Graham Perrin <grahamperrin@gmail.com>
MFC after: 3 days
lib/csu/tests/dynamiclib requires libh_csu.so be built first. I'm not
sure this is the most correct/best way to address this but it solves
the issue in my testing.
PR: 233734
Sponsored by: The FreeBSD Foundation
As it does for recv*(2), MSG_DONTWAIT indicates that the call should
not block, returning EAGAIN instead. Linux and OpenBSD both implement
this, so the change makes porting easier, especially since we do not
return EINVAL or so when unrecognized flags are specified.
Submitted by: Greg V <greg@unrelenting.technology>
Reviewed by: tuexen
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18728
When presented with an arg string like '-l-', getopt_long will successfully
parse out the 'l' short option, then proceed to match '--' against the first
longopts entry as it later does a strncmp with len=0. This latter bit is
arguably another bug in itself, but presumably not a practical issue as all
callers of parse_long_options are already doing the right thing (except this
one pointed out).
An opt string like '-l-' should be considered malformed and throw a bad
argument rather than behaving as if '--' were passed. It cannot possibly do
what the invoker expects, and it's probably the result of a typo (ls -l- a)
rather than any intent.
Reported by: Tony Overfield <toverfield@yahoo.com>
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D18616
Add a short description of the function to the appropriate man page and add
reference to it where it makes sense.
Reviewed by: bcr, markj, 0mp
Approved by: markj
Differential Revision: https://reviews.freebsd.org/D18725
This merge brings in a couple new files, which needed to be attached to the
build; a new dependency on <limits.h>, which must be stubbed; and a name
change in the Context parameter constants, from ZSTD_p_foo to ZSTD_c_foo.
Significantly, it fixes a kernel build error with GCC where floating-point
functions were included in the kernel build, by hiding them under the same
compile-time #ifdef that already covered their invocation. That issue was
introduced to FreeBSD in the 1.3.7 update and tracked upstream here:
https://github.com/facebook/zstd/issues/1386
The full 1.3.8 release notes can be found on Github:
https://github.com/facebook/zstd/releases/tag/v1.3.8
Relnotes: yes
