ASAN reports become a lot more useful with llvm-symbolizer in $PATH, and the
build is not much more time-consuming. The added benefit is that the
resulting reports will actually include symbol information; without, thread
trace information includes a bunch of addresses that immediately resolve to
an inline function in
^/contrib/compiler-rt/lib/sanitizer_common/sanitizer_common.h and take a
little more effort to examine.
Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20484
This 'r' case should have belonged to the switch in the first place, but
I had somehow missed the switch when initially adding the rgrep link. The
zgrep script later came along and faithfully left this case standing alone,
so we will now go ahead and join it.
Nearby comment also adjusted a tad bit for wording and style.
Reported by: Daniel Ebdrup
MFC after: 3 days
See usage for the command line structure. Man page will come shortly.
Reviewed by: jilles, tmunro
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D20258
Sync libarchive with vendor.
Relevant vendor changes:
Issue #795: XAR - do not try to add xattrs without an allocated name
PR #812: non-recursive option for extract and list
PR #958: support reading metadata from compressed files
PR #999: add --exclude-vcs option to bsdtar
Issue #1062: treat empty archives with a GNU volume header as valid
PR #1074: Handle ZIP files with trailing 0s in the extra fields
(Android APK archives)
PR #1109: Ignore padding in Zip extra field data (Android APK archives)
PR #1167: fix problems related to unreadable directories
Issue #1168: fix handling of strtol() and strtoul()
PR #1172: RAR5 - fix invalid window buffer read in E8E9 filter
PR #1174: ZIP reader - fix of MSZIP signature parsing
PR #1175: gzip filter - fix reading files larger than 4GB from memory
PR #1177: gzip filter - fix memory leak with repeated header reads
PR #1180: ZIP reader - add support for Info-ZIP Unicode Path Extra Field
PR #1181: RAR5 - fix merge_block() recursion
(OSS-Fuzz 12999, 13029, 13144, 13478, 13490)
PR #1183: fix memory leak when decompressing ZIP files with LZMA
PR #1184: fix RAR5 OSS-Fuzz issues 12466, 14490, 14491, 12817
OSS-Fuzz 12466: RAR5 - fix buffer overflow when parsing huffman tables
OSS-Fuzz 14490, 14491: RAR5 - fix bad shift-left operations
OSS-Fuzz 12817: RAR5 - handle a case with truncated huffman tables
PR #1186: RAR5 - fix invalid type used for dictionary size mask
(OSS-Fuzz 14537)
PR #1187: RAR5 - fix integer overflow (OSS-Fuzz 14555)
PR #1190: RAR5 - RAR5 don't try to unpack entries marked as directories
(OSS-Fuzz 14574)
PR #1196: RAR5 - fix a potential SIGSEGV on 32-bit builds
OSS-Fuzz 2582: RAR - fix use after free if there is an invalid entry
OSS-Fuzz 14331: RAR5 - fix maximum owner name length
OSS-Fuzz 13965: RAR5 - use unsigned int for volume number + range check
Additional RAR5 reader changes:
- support symlinks, hardlinks, file owner, file group, versioned files
- change ARCHIVE_FORMAT_RAR_V5 to 0x100000
- set correct mode for readonly directories
- support readonly, hidden and system Windows file attributes
MFC after: 2 weeks
Historically we have not distinguished between kernel wirings and user
wirings for accounting purposes. User wirings (via mlock(2)) were
subject to a global limit on the number of wired pages, so if large
swaths of physical memory were wired by the kernel, as happens with
the ZFS ARC among other things, the limit could be exceeded, causing
user wirings to fail.
The change adds a new counter, v_user_wire_count, which counts the
number of virtual pages wired by user processes via mlock(2) and
mlockall(2). Only user-wired pages are subject to the system-wide
limit which helps provide some safety against deadlocks. In
particular, while sources of kernel wirings typically support some
backpressure mechanism, there is no way to reclaim user-wired pages
shorting of killing the wiring process. The limit is exported as
vm.max_user_wired, renamed from vm.max_wired, and changed from u_int
to u_long.
The choice to count virtual user-wired pages rather than physical
pages was done for simplicity. There are mechanisms that can cause
user-wired mappings to be destroyed while maintaining a wiring of
the backing physical page; these make it difficult to accurately
track user wirings at the physical page layer.
The change also closes some holes which allowed user wirings to succeed
even when they would cause the system limit to be exceeded. For
instance, mmap() may now fail with ENOMEM in a process that has called
mlockall(MCL_FUTURE) if the new mapping would cause the user wiring
limit to be exceeded.
Note that bhyve -S is subject to the user wiring limit, which defaults
to 1/3 of physical RAM. Users that wish to exceed the limit must tune
vm.max_user_wired.
Reviewed by: kib, ngie (mlock() test changes)
Tested by: pho (earlier version)
MFC after: 45 days
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D19908
This is a prerequisite of unifying kernel zlib instances.
Submitted by: Yoshihiro Ota <ota at j.email.ne.jp>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D20191
During processing we maintain symbol offsets in the 64-bit s_so array,
and when writing the archive convert to 32-bit if no offsets are greater
than 4GB. However, this was somewhat inefficient as we looped over the
array twice: first, converting to big endian and second, writing each
32-bit value one at a time (and incorrectly so on big-endian platforms).
Instead, when writing a 32-bit archive shuffle convert symbol data to
big endian (as required by the ar format) and shuffle to the beginning
of the allocation at the same time.
Also correct emission of the symbol count on big endian platforms.
Further changes are planned, but this should fix powerpc64.
Reported by: jhibbits, mlinimon
Reviewed by: jhibbits, Gerald Aryeetey (earlier)
Tested by: jhibbits
MFC after: 10 days
MFC with: r346079
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20007
This change allows the user to once again override the C++ standard, restoring
high-level pre-r345708 behavior.
This also unbreaks building lib/ofed/libibnetdisc/Makefile with a non-C++11
capable compiler, e.g., g++ 4.2.1, as the library supported being built with
older C++ standards.
MFC after: 2 weeks
MFC with: r345708
Reviewed by: emaste
Reported by: jbeich
Differential Revision: https://reviews.freebsd.org/D19895 (as part of a larger change)
Previous spellings of my name (NGie, Ngie) weren't my legal spelling. Use Enji
instead for clarity.
While here, remove "All Rights Reserved" from copyrights I "own".
MFC after: 1 week
This is somewhat more readable than pointer arithmetic. Also remove an
unnecessary cast while here.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
This is a minor simplification; if we do not have any symbols the empty
symbol table can be in 32-bit format.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Add a stat counter to track ipv6 atomic fragments. Atomic fragments can be
generated in response to invalid path MTU values, but are also a potential
attack vector and considered harmful (see RFC6946 and RFC8021).
While here add tracking of the atomic fragment counter to netstat and systat.
Reviewed by: tuexen, jtl, bz
Approved by: jtl (mentor), bz (mentor)
Event: Aberdeen hackathon 2019
Differential Revision: https://reviews.freebsd.org/D17511
This allows an endless stream of random data within the given bounds.
It already worked if a seed was provided as the 4th argument but not
if one was left out.
In collaboration with: jhb
MFC after: 2 weeks
Relnotes: yes
Experimentally, reduces sort -R time of a 148160 line corpus from about
3.15s to about 0.93s on this particular system.
There's probably room for improvement using some digest other than md5, but
I don't want to look at sort(1) anymore. Some discussion of other possible
improvements in the Test Plan section of the Differential.
PR: 230792
Reviewed by: jhb (earlier version)
Differential Revision: https://reviews.freebsd.org/D19885
Bound input file processing length to avoid the issue reported in [1]. For
simplicity, only allow regular file and character device inputs. For
character devices, only allow /dev/random (and /dev/urandom symblink).
32 bytes of random is perfectly sufficient to seed MD5; we don't need any
more. Users that want to use large files as seeds are encouraged to truncate
those files down to an appropriate input file via tools like sha256(1).
(This does not change the sort algorithm of sort -R.)
[1]: https://lists.freebsd.org/pipermail/freebsd-hackers/2018-August/053152.html
PR: 230792
Reported by: Ali Abdallah <aliovx AT gmail.com>
Relnotes: yes
There's no reason to order based on strcmp of ASCII digests instead of
memcmp of the raw digests.
While here, remove collision fallback. If you collide two MD5s, they're
probably the same string anyway. If robustness against MD5 collisions is
desired, maybe we shouldn't use MD5.
None of the behavior of sort -R is specified by POSIX, so we're free to
implement this however we like. E.g., using a 128-bit counter and block cipher
to generate unique indices for each line of input.
PR: 230792 (2/many)
Relnotes: This will change the sort order for a given dataset with a
given seed. Other similarly breaking changes are planned.
Sponsored by: Dell EMC Isilon
from 1.0.0:
Add "continuation" flag, to allow multiple "xo" invocations in a single line of output (#58)
Add --top-wrap to make top-level JSON wrappers
Add --{open,close}-{list,instace} options
Add xo_xml_leader(), to detect use of some bogus XML tags. It's still bad form, but it's a little safer now
Avoid call to xo_write before xo_flush, since the latter calls the former
Check return code from xo_flush_h properly (<0) (FreeBSD Bug 236935)
For JSON output, avoid newline before a container's close brace (#62)
Merge branch 'text_only' of https://github.com/zvr/libxo into zvr-text_only
Use XO_USE_INT_RETURN_CODES, not USE_INT_RETURN_CODES
add docs for --continuation
add docs for --not-first
call xo_state_set_flags before values and close containers; add XOIF_MADE_OUTPUT flag to track state; make proper empty JSON objects in xo_finish
color_map code has to be #ifdef'd out, since the struct definition
correct xo_flush_func_t (doesn't use xo_ssize_t)
make depth change for --top-wrap only for JSON
fix to handle --top-wrap in "xo" by being more consistent with handling trailing newlines
fix to handle text-only version #64 (from zvr)
fix xo_buf_has_room for round up to the next XO_BUFSIZ, not just add XO_BUFSIZ to the size (FreeBSD Bug 236937)
update docs for new "xo" options
update functions to use xo_ssize_t
update test cases
from 1.0.1:
Add EINTEGRITY to .pot files under test/gettext/ (fix from FreeBSD)
from 1.0.2:
handle failure from xo_vnsprintf; don't add -1 to "rc"
PR: 236937, 236935
Submitted by: phil
Reported by: Alfonso S. Siciliano <alfix86@gmail.com>
MFC after: 2 weeks
The value was changed from int to bool. Since the new type
is smaller, the rest of the variable in the caller was left
unitialized.
PR: 236714
Reported by: trasz
Diagnosed by: markj
Sponsored by: The FreeBSD Foundation
to no longer be displayed. This was because the reimplementation of
setup_buffer() did not copy the previous contents into any reallocated
buffer.
Reported by: James Wright <james.wright@jigsawdezign.com>
PR: 236947
MFC after: 3 days
CXXSTD was added as the C++ analogue to CSTD.
CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.
This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.
As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.
Notes:
This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.
Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:
Before this commit:
```
CXXFLAGS+= -std=c++14
```
After this commit:
```
CXXSTD= c++14
```
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 month
MFC with: r345203, r345704, r345705
Relnotes: yes
Tested with: make tinderbox
Differential Revision: https://reviews.freebsd.org/D19732
When a review is closed via Phabricator it updates the patch attached to the
review. I downloaded the raw patch from Phabricator, applied it, and repeated
my mistake from r345704 by accident mixing content from D19732 and D19738.
For my own personal sanity, I will try not to mix reviews like this in the
future.
MFC after: 1 month
MFC with: r345706
Approved by: emaste (mentor, implicit)
CXXSTD was added as the C++ analogue to CSTD.
CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.
This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.
As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.
Notes:
This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.
Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:
Before this commit:
```
CXXFLAGS+= -std=c++14
```
After this commit:
```
CXXSTD= c++14
```
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 month
MFC with: r345203, r345704, r345705
Relnotes: yes
Tested with: make tinderbox
Differential Revision: https://reviews.freebsd.org/D19732
I accidentally committed code from two reviews. I will reintroduce the code to
bsd.progs.mk as part of a separate commit from r345704.
Approved by: emaste (mentor, implicit)
MFC after: 2 months
MFC with: r345704
CXXSTD defaults to `-std=c++11` with supporting compilers; `-std=gnu++98`,
otherwise for older versions of g++.
This change standardizes the CXXSTD variable, originally added to
googletest.test.inc.mk as part of r345203.
As part of this effort, convert all `CXXFLAGS+= -std=*` calls to use `CXXSTD`.
Notes:
This value is not sanity checked in bsd.sys.mk, however, given the two
most used C++ compilers on FreeBSD (clang++ and g++) support both modes, it is
likely to work with both toolchains. This method will be refined in the future
to support more variants of C++, as not all versions of clang++ and g++ (for
instance) support C++14, C++17, etc.
Any manual appending of `-std=*` to `CXXFLAGS` should be replaced with CXXSTD.
Example:
Before this commit:
```
CXXFLAGS+= -std=c++14
```
After this commit:
```
CXXSTD= c++14
```
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 1 month
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D19732
Highlights:
- Bugfix for order in which /delete-node/ and /delete-property/ are
processed [0]
- /omit-if-no-ref/ support has been added (used only by U-Boot at this
point, in theory)
- GPL dtc compat version bumped to 1.4.7
- Various small fixes and compatibility improvements
Reported by: strejda [0]
MFC after: 1 week
exist
Apply EX_UNAVAILABLE patch part from PR 170775 to match the documentation.
Checked with a command from PR 210770:
lockf -n /tmp/doesnotexist echo; echo $?
PR: 210770
MFC after: 1 week
- Fix markup.
- Mention that process can only allow tracing for itself. This is already
stated in procctl(2), but requiring knowledge of the syscall description
is too much for the tool user.
- Clearly state that query mode only works for existing process.
Noted and reviewed by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
r344504 added an extra ARP_LOG() call in case of an if_output() failure.
It turns out IPv4 can be noisy. In order to not spam the console by default:
(a) add a counter for these events so people can keep better track of how
often it happens, and
(b) add a sysctl to select the default ARP_LOG log level and set it to
INFO avoiding the one (the new) DEBUG level by default.
Claim a spare (1st one after 10 years since the stats were added) in order
to not break netstat from FreeBSD 12->13 updates in the future.
Reviewed by: karels
Differential Revision: https://reviews.freebsd.org/D19490
This matches GNU seq, for example.
For users that are looking for similar functionality, 'jot -b foo N' will
print 'foo' N times. See jot(1).
PR: 236347
Reported by: <y AT maya.st>
Sponsored by: Dell EMC Isilon
The ports version of cal is an abandonware so in order to minimize the
potential bit rot of our documentation let's not mention it at all.
Interested users are going to find suitable alternatives anyway on their
own.
Reported by: bapt
Approved by: bapt (src)
Approved by: krion (mentor, implicit), mat (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D19492
Without this, if output is redirected from the console, it is buffered for
too long, making tool quite unusable.
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Set the number of partitions entries in the GPT header to a
multiple of the number of entries that fit in a sector.
PR: 236238
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D19465
Note that only entries wired by userspace are shown as such. In
particular, entries transiently wired by sysctl_wire_old_buffer() are
not flagged as wired in procstat -v output.
Reviewed by: kib (previous version)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D19461
These are taken directly from the density report from a TS1160
tape drive. (Using mt getdensity)
A TS1160 drive stores 20TB raw (60TB with compression) on a JE tape.
lib/libmt/mtlib.c:
Add 3592A6 encrypted/unencrypted density codes, and bpmm/bpi
values.
usr.bin/mt/mt.1:
Add 3592B5 encrypted/unencrypted density codes, bpmm/bpi
values and number of tracks. Bump the man page date.
MFC after: 3 days
Sponsored by: Spectra Logic
Buildworld failed when both WITHOUT_INET6_SUPPORT and INET equivalent were set.
Fix netstat and syslogd by applying appropriate #ifdef INET/INET6 to make world
compile again.
Reviewed by: ngie, hrs, ume
Welcomed by: Michael Dexter (D17040)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D19418
This change is made in the name of GNU patch compatibility. If GNU patch is
fed a zero-length patch, it will exit successfully with no output. This is
used in at least one port to date (comms/wsjtx), and we break on this usage.
It seems unlikely that anyone relies on patch(1) calling their completely
empty patch garbage and failing, and GNU compatibility is a plus if it helps
with porting, so make the switch.
Reported by: db
MFC after: 2 weeks
The default handling showed the argument as hex. Add explicit handling so
we can show it as decimal, since that's how we show file descriptors
everywhere else.
Approved by: mjg (mentor)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D19295
Split the rights-limiting code into two cases: if one of the input
files isn't a regular file, use caph_limit_stream(3) instead of
open-coding the same logic; if both input files are regular files,
and the initial attempts to map them succeed, we limit the rights on
those files to CAP_MMAP_R.
Add a regression test for PR 234885.
PR: 234885
Reviewed by: delphij
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19216
Subversion builds and links against its own .a archives using local
rules, so did not benefit from with the WITH_PIE library support added
in r344179. Apply the same _pie suffix locally.
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D19246
In line_update(), set lastcol correctly after moving to any non-zero
column, so the "overwrite old stuff" part does not attempt to address
negative offsets in the current line.
Rewrite setup_buffer() to always allocate at least 80 characters,
otherwise various calls to summary_format() will overwrite the end of
the buffers, if the screen width gets small enough.
MFC after: 1 week
calendar(1) can have input in various encoding, specifying
LANG=<locale_name> to enable calendar(1) to determine which one to use.
The problem is the content of the calendar itself is exposed as is making it
unreadable in many cases. For example french calendar which is encoded
ISO8859-1 is rendered badly in a fr_FR.UTF-8 environment.
Using iconv allows to solve this issue.
This will also allow to keep only 1 encoding in base for those files without
breaking user existing setup
Reported by: many
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D19221
Comment for CAPFAIL_LOOKUP refered only to paths containing ".." but
it is returned for other restricted VFS lookup cases, such as absolute
paths or openat(AT_FDCWD, ...).
Building binaries as PIE allows the executable itself to be loaded at a
random address when ASLR is enabled (not just its shared libraries).
With this change PIE objects have a .pieo extension and INTERNALLIB
libraries libXXX_pie.a.
MK_PIE is disabled for some kerberos5 tools, Clang, and Subversion, as
they explicitly reference .a libraries in their Makefiles. These can
be addressed on an individual basis later. MK_PIE is also disabled for
rtld-elf because it is already position-independent using bespoke
Makefile rules.
Currently only dynamically linked binaries will be built as PIE.
Discussed with: dim
Reviewed by: kib
MFC after: 1 month
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18423
With this change, randomization can be enabled for all non-fixed
mappings. It means that the base address for the mapping is selected
with a guaranteed amount of entropy (bits). If the mapping was
requested to be superpage aligned, the randomization honours the
superpage attributes.
Although the value of ASLR is diminshing over time as exploit authors
work out simple ASLR bypass techniques, it elimintates the trivial
exploitation of certain vulnerabilities, at least in theory. This
implementation is relatively small and happens at the correct
architectural level. Also, it is not expected to introduce
regressions in existing cases when turned off (default for now), or
cause any significant maintaince burden.
The randomization is done on a best-effort basis - that is, the
allocator falls back to a first fit strategy if fragmentation prevents
entropy injection. It is trivial to implement a strong mode where
failure to guarantee the requested amount of entropy results in
mapping request failure, but I do not consider that to be usable.
I have not fine-tuned the amount of entropy injected right now. It is
only a quantitive change that will not change the implementation. The
current amount is controlled by aslr_pages_rnd.
To not spoil coalescing optimizations, to reduce the page table
fragmentation inherent to ASLR, and to keep the transient superpage
promotion for the malloced memory, locality clustering is implemented
for anonymous private mappings, which are automatically grouped until
fragmentation kicks in. The initial location for the anon group range
is, of course, randomized. This is controlled by vm.cluster_anon,
enabled by default.
The default mode keeps the sbrk area unpopulated by other mappings,
but this can be turned off, which gives much more breathing bits on
architectures with small address space, such as i386. This is tied
with the question of following an application's hint about the mmap(2)
base address. Testing shows that ignoring the hint does not affect the
function of common applications, but I would expect more demanding
code could break. By default sbrk is preserved and mmap hints are
satisfied, which can be changed by using the
kern.elf{32,64}.aslr.honor_sbrk sysctl.
ASLR is enabled on per-ABI basis, and currently it is only allowed on
FreeBSD native i386 and amd64 (including compat 32bit) ABIs. Support
for additional architectures will be added after further testing.
Both per-process and per-image controls are implemented:
- procctl(2) adds PROC_ASLR_CTL/PROC_ASLR_STATUS;
- NT_FREEBSD_FCTL_ASLR_DISABLE feature control note bit makes it possible
to force ASLR off for the given binary. (A tool to edit the feature
control note is in development.)
Global controls are:
- kern.elf{32,64}.aslr.enable - for non-fixed mappings done by mmap(2);
- kern.elf{32,64}.aslr.pie_enable - for PIE image activation mappings;
- kern.elf{32,64}.aslr.honor_sbrk - allow to use sbrk area for mmap(2);
- vm.cluster_anon - enables anon mapping clustering.
PR: 208580 (exp runs)
Exp-runs done by: antoine
Reviewed by: markj (previous version)
Discussed with: emaste
Tested by: pho
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D5603
Avoid setting zero bytes beyond the length of the 'thisline' parameters
in i_process() and u_process(), and don't attempt to memset a negative
number of bytes.
MFC after: 1 week
Suggest using -a to egrep to properly see executed commands.
This is a minor improvement to the manpage. A better improvement
would be removal or gigantic warnings.
Sponsored by: Dell EMC
MFC after: 1 week
For example, from the referenced PR [1]:
$ mkdir /tmp/lib/ /tmp/libexec
$ touch /tmp/lib/foo.so
$ install -lrs /tmp/lib/foo.so /tmp/libexec/
The common path identification bits terminate src at /tmp/lib/ and the
destination at /tmp/libe. The subsequent backtracking is then incorrect, as
it traverses the destination and backtraces exactly one level while eating
the 'libexec' because it was previously (falsely) identified as common with
'lib'.
The obvious fix would be to make sure we've actually terminated just after
directory separators and rewind a character if we haven't. In the above
example, we would end up rewinding to /tmp/ and subsequently doing the right
thing.
Test case added.
PR: 235330 [1]
MFC after: 1 week
- Use "in" instead of "on" when referring to directory and UFS partition.
- Switch from hw.physmem to hw.realmem and add a description to
distinguish the two.
- Explain why the "df" command is having trouble displaying ZFS sizes
correctly. Add a bit more descriptive text to help why the output of
"zfs list -o space" should be used.
- Switch to vmstat instead of iostat display for systat(1) as it shows
more information on one screen. Describe what is displayed based on the
text of the man page. Change the list of the other values accordingly.
- Sort the flags to "zfs destroy" alphabetically.
Reviewed by: rgrimes
Approved by: rgrimes
MFC after: 8 days
Differential Revision: https://reviews.freebsd.org/D18993
find(1) ignores -type w passed to it. With this patch find(1) properly
identifies and prints whiteouts.
PR: 126384, 156703
Submitted by: oleg@mamontov.net
MFC after: 1 week
While these warnings are false positives, the use of strdup() instead of
malloc() and strcpy() simplifies and clarifies the code.
While checking the remaining uses of strcpy and strcat I noticed an
assignment of a strlen() to a variable "s", whose value needs to be
preserved for use in later output routines (where it is used to allocate
a buffer). I do not think that the value of "s" will come out lower than
its correct value and thus there is no risk of a buffer overflow, in the
general case, but a specially crafter argument might lead to an overflow.
The bogus assignment to "s" is removed since this value was only used a
single time in the following malloc() call, which has been removed.
MFC after: 2 weeks
Add a bunch of examples on how to use ZFS features like:
- listing available space,
- setting and displaying a userquota,
- displaying pool I/O statistics and pool history,
- displaying the compression ratio for a dataset,
- various list options (sorting, removing headers),
- performing a dry-run of a snapshot delete,
- removing a range of snapshots,
- setting a custom property,
- preventing removal of a snapshot with ZFS holds,
- permission sets for zfs send/receive.
Additionally, clarify the existing examples a bit when
it comes to displaying space by mentioning UFS explicitly.
Other examples include displaying I/O in top(1), querying
sysctl(8) for active CPUs and available RAM. Mention systat(1)
and its options, too.
While here, reformat the example to upload a dmesg(8) a bit
to wrap properly.
Thanks to Allan Jude for his help with some of the ZFS examples.
Reviewed by: dru,allanjude
Approved by: allanjude (earlier version)
MFC after: 3 days
Relnotes: yes (ZFS examples in freebsd-tips)
Differential Revision: https://reviews.freebsd.org/D18541
We already call setutxent() once during initialization. Furthermore,
the subsequent calls occur after the process has entered capability
mode, so they fail, and attempts to fetch database entries fail as
a result.
PR: 235096
Submitted by: fullermd@over-yonder.net
MFC after: 3 days
According to systat(1) :only option is supposed to accept multiple drives
but the parser for its arguments stops after first entry. Fix the parser
logic to accept multiple drives.
PR: 59220
Reported by: Andy Farkas <andyf@speednet.com.au>
MFC after: 1 week
Test failures don't seem to propagate up if atf_check is run in
a pipeline. Thus, the tests continued to pass despite the bug reverted
in r343245.
MFC after: 1 week
It breaks the special mode specified by passing "-" as one of the
input files. Revert for now while we discuss a fix.
PR: 234885
Reported by: delphij
MFC after: now
getopt_long(3) requires the long options be terminated by a NULL block.
Without the terminator, an invalid long option results in a segmentation
fault.
Reported by: Brandon Bergren
MFC after: 1 week
This is mostly a style fix since the code in question is not called multiple
times and doesn't have cummulative effect.
PR: 204953
Submitted by: David Binderman <dcb314@hotmail.com>
MFC after: 1 week
- Limit rights on stdio before opening input files. Otherwise, open()
may return one of the standard descriptors and we end up limiting
rights such that we cannot read from one of the input files.
- Use caph_limit_stdio(), which suppresses EBADF, to ensure that
we don't emit an error if one of the stdio streams is closed.
- Don't bother further limiting rights on stdin when stdin isn't going
to be used. Doing so correctly requires checking for a number of
edge cases, and it doesn't provide any significant benefit.
PR: 234885
Reviewed by: oshogbo
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18860
bc.y: Rev 1.50
- write parse errors to stderr, prompted by Martijn Dekker
- we're only interactive if stdout en stderr are a tty as well as stdin
PR: 234430
Obtained from: OpenBSD
MFC after: 1 week
The goal of this change is to make it easier to use getconf to query
the number of available processors.
Sadly it's unclear per POSIX, which form (with a preceding _ or
lacking it) is correct. I will bring this up on the Austin Group list so
this point is clarified for implementors that might rely on this getconf
variable in future POSIX spec versions.
This is something I noticed when trying to import GoogleTest to FreeBSD
as one of the CI scripts uses this variable on Linux.
MFC after: 2 weeks
Approved by: emaste (mentor)
Differential Revision: https://reviews.freebsd.org/D18640
The default build should install bsdgrep(1) and grep(1), with the latter
being gnugrep(1). WITH_BSD_GREP flips this situation such that we have
gnugrep(1) and grep(1), with the latter being bsdgrep(1).
Changes to start installing the zgrep script out of usr.bin/grep
inadvertently altered the default build such that grep(1) was being
installed, and it was bsdgrep(1). Correct the typo.
Reported by: bcran
MFC after: 3 days
This merge brings in a couple new files, which needed to be attached to the
build; a new dependency on <limits.h>, which must be stubbed; and a name
change in the Context parameter constants, from ZSTD_p_foo to ZSTD_c_foo.
Significantly, it fixes a kernel build error with GCC where floating-point
functions were included in the kernel build, by hiding them under the same
compile-time #ifdef that already covered their invocation. That issue was
introduced to FreeBSD in the 1.3.7 update and tracked upstream here:
https://github.com/facebook/zstd/issues/1386
The full 1.3.8 release notes can be found on Github:
https://github.com/facebook/zstd/releases/tag/v1.3.8
Relnotes: yes
BSD ar currently does not support the /SYM64/ 64-bit symbol table, and
previously truncated to 32-bits, silently producing corrupted archives
larger than 4GB.
This is another overflow case in addtion to r342575.
PR: 234454
Reported by: Aijaz Baig, imp
MFC after: 2 weeks
MFC with: r342575
Sponsored by: The FreeBSD Foundation
BSD ar currently does not support the /SYM64/ 64-bit symbol table, and
previously truncated to 32-bits, silently producing corrupted archives
larger than 4GB.
Note that this is only a partial fix; additional checks will come.
PR: 234454
Reported by: Aijaz Baig, imp
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
- Add missing parameters to flags in the description of available options.
- Remove spaces between alternative parameters and "|".
- Align descriptions of options to the longest option.
- Use em dash instead of a hyphen.
Reviewed by: bcr
Approved by: doc (bcr)
Approved by: krion (mentor, implicit), mat (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D18608
This fixes truss when built as part of a riscv64sf world. Additionally,
if FreeBSD ever supports RV32 RISC-V most of this file can be used as-is
just as a single file is used for all of the MIPS ABIs.
Sponsored by: DARPA
Summary:
Currently, truss doesn't work on ELFv2 processes due to not recognizing the ABI.
Since there's no special handling needed here, just adding a PROCABI struct for
it is sufficient to get it working.
Submitted by: git_bdragon.rtk0.net
Differential Revision: https://reviews.freebsd.org/D18352
Some options come from static constructors in LLVM libraries and are
automatically added to llvm's usage output. They're not really supposed
to be llvm-objdump options.
Reported by: Fangrui Song in LLVM review D54864
Sponsored by: The FreeBSD Foundation
Some options appear in llvm-objdump's usage information as a side effect
of its option parsing implementation and are not actually llvm-objdump
options. Reported in LLVM review https://reviews.llvm.org/D54864.
Reported by: Fangrui Song
Sponsored by: The FreeBSD Foundation
Based on llvm-objdump's online documentation and usage information.
This serves as a starting point; additional detail and cleanup still
required.
Also being submitted upstream in LLVM review D54864. I expect to use
this bespoke copy while we have LLVM 6.0 or 7.0 in FreeBSD; when we
update to LLVM 8.0 it should be upstream and we will switch to it.
PR: 233437
Reviewed by: bcr (man formatting)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18309
Under some conditions pom would report "waning" and then "full", show
higher percentages than it should, and get confused by DST. Fix.
Before:
2018.01.30: The Moon is Waxing Gibbous (97% of Full)
2018.01.31: The Moon is Waning Gibbous (100% of Full)
2018.02.01: The Moon is Full
2018.02.02: The Moon is Waning Gibbous (98% of Full)
After:
2018.01.30: The Moon is Waxing Gibbous (96% of Full)
2018.01.31: The Moon is Waxing Gibbous (99% of Full)
2018.02.01: The Moon is Full
2018.02.02: The Moon is Waning Gibbous (97% of Full)
PR: 231705
Submitted by: Andrew Gierth
Approved by: allanjude (mentor)
MFC after: 2 weeks
Differential Revision: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231705
NYCBUG database.
We want to encourage our users to upload their dmesgs so that the project can
get a better insight into what kind of hardware is run on. This helps in making
data-driven decisions about i.e., platform and driver support.
Note that dmesgs may contain sensitive information like hardware serial numbers,
hence uploading them without review is discouraged.
Reviewed by: brooks, imp, allanjude
Approved by: allanjude
MFC after: 5 days
Differential Revision: https://reviews.freebsd.org/D17705
Several statistic counters are uint64_t values and are printed by systat
using %lu. This results in displaying wrong numbers. Use PRIu64 instead.
While there, print variables of size_t using %zd.
MFC after:i 3 days
Differential Revision: https://reviews.freebsd.org/D17838
llvm-profdata is used with llvm-cov for code coverage (although llvm-cov
can also operate independently in a gcov-compatible mode).
Although llvm-profdata can be used independently of llvm-cov it makes
sense to group these under one option.
Also handle these in OptionalObsoleteFiles.inc while here.
Sponsored by: The FreeBSD Foundation
llvm-cov provides a gcov-compatible interface when invoked as gcov.
Reviewed by: dim, markj
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17923
Assume that user wants to run with capsicum support if he builds the software
with HAVE_CAPSICUM. Treat running application without capsicum in the kernel as
an error.
MFC after: 3 weeks
While here, use LANG as the proper source to select man pages language/encoding,
falling back to LC_CTYPE.
Reviewed by: bapt
Approved by: kib (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D17835
Currently truss(1) shows shm_open(SHM_ANON, ...) as shm_open("(null)", ...).
Detect the special value and display it by name.
Reviewed by: jhb, allanjude, tuexen
Approved by: mjg (mentor)
MFC with: r339224
Differential Revision: https://reviews.freebsd.org/D17461
for the known conflicts ("control" characters can't be "print"able).
POSIX doesn't explicitly forbid this, and actually includes <space>
character in "print".
PR: 225692
Reviewed by: bapt, cem (previous version), pfg (previous version)
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D17467
Previously ar would report an error like "ar: fatal: Write error"
without including additional errno information. Change warnings and
errors to include archive_errno() so that the user may have some idea
of the reason for the failure.
Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17650
Support a "live" mode in ktrdump enabled via the -l flag. In this
mode, ktrdump polls the kernel's trace buffer periodically (currently
hardcoded as a 50 millisecond interval) and dumps any newly added
entries. Fancier logic for the timeout (e.g. a command line option or
some kind of backoff based on the time since the last entry) can be
added later as the need arises.
While here, fix some bugs from when this was Capsicum-ized:
- Use caph_limit_stream() for the output stream so that isatty() works
and the output can be line-buffered (especially useful for live
mode).
- Use caph_limit_stderr() to permit error messages to be displayed if
an error occurs after cap_enter().
Reviewed by: kib, 0mp (manpage)
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D17315
it appropriately when building share/ctypedef and share/colldef.
This makes the resulting locale data in EL->EB (amd64->powerpc64) cross
build and in the native EB build match. Revert the changes done to libc
in r308170 as they are no longer needed.
PR: 231965
Reviewed by: bapt, emaste, sbruno, 0mp
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D17603
Due to markup issues, the DESCRIPTION OF MEMORY section is rather
unreadable; rework it a bit, using subsections for different lines of the
top output, and move it closer to description.
While here, pet manlint ordering other sections as expected.
Submitted by: Yuri Pankov <yuripv@yuripv.net>
Reviewed by: eadler
Approved by: re (gjb), krion (mentor)
Differential Revision: https://reviews.freebsd.org/D17369
This is a direct commit to a generated file. Simon plans to fix this
upstream before the next import.
PR: 231557
Approved by: re (gjb)
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
This leverages CONFS to handle the install and purges an old comment.
Approved by: re (blanket, pkgbase), bapt (mentor)
Differential Revision: https://reviews.freebsd.org/D17215
Update libarchive to 3.3.3
As all important changes have already been merged from libarchive git
this is just version number bump, documentation update and some
polishing for cpio tests. Other source code changes are not relevant to
FreeBSD.
Approved by: re (gjb)
MFC after: 1 week
We generate the installed objcopy man page from ELF Tool Chain's
elfcopy, but the sed expresion used for this ended up producing
"objcopy, objcopy - copy and translate object files".
Instead of replacing the first "elfcopy" with objcopy, just remove it.
Approved by: re (gjb)
imprudent reuse of static buffers, the end-of-transfer statistics
displayed when stdout is not a tty always ended up as 0 B / 0 Bps.
Reorganize the code to use caller-provided buffers, tweak the ETA
display a bit, and reduce the visual differences between the tty and
non-tty end-of-transfer displays.
PR: 202424
Approved by: re (gjb@)
Don't just exit when encountering the 'q' command if we edit file
inplace, and give mf_fgets() a chance to actually handle the
inplace case.
Also add a regression test.
Submitted by: Yuri Pankov <yuripv@yuripv.net>
Approved by: re (kib)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D16798
Previously, x86 used static ranges of IRQ values for different types
of I/O interrupts. Interrupt pins on I/O APICs and 8259A PICs used
IRQ values from 0 to 254. MSI interrupts used a compile-time-defined
range starting at 256, and Xen event channels used a
compile-time-defined range after MSI. Some recent systems have more
than 255 I/O APIC interrupt pins which resulted in those IRQ values
overflowing into the MSI range triggering an assertion failure.
Replace statically assigned ranges with dynamic ranges. Do a single
pass computing the sizes of the IRQ ranges (PICs, MSI, Xen) to
determine the total number of IRQs required. Allocate the interrupt
source and interrupt count arrays dynamically once this pass has
completed. To minimize runtime complexity these arrays are only sized
once during bootup. The PIC range is determined by the PICs present
in the system. The MSI and Xen ranges continue to use a fixed size,
though this does make it possible to turn the MSI range size into a
tunable in the future.
As a result, various places are updated to use dynamic limits instead
of constants. In addition, the vmstat(8) utility has been taught to
understand that some kernels may treat 'intrcnt' and 'intrnames' as
pointers rather than arrays when extracting interrupt stats from a
crashdump. This is determined by the presence (vs absence) of a
global 'nintrcnt' symbol.
This change reverts r189404 which worked around a buggy BIOS which
enumerated an I/O APIC twice (using the same memory mapped address for
both entries but using an IRQ base of 256 for one entry and a valid
IRQ base for the second entry). Making the "base" of MSI IRQ values
dynamic avoids the panic that r189404 worked around, and there may now
be valid I/O APICs with an IRQ base above 256 which this workaround
would incorrectly skip.
If in the future the issue reported in PR 130483 reoccurs, we will
have to add a pass over the I/O APIC entries in the MADT to detect
duplicates using the memory mapped address and use some strategy to
choose the "correct" one.
While here, reserve room in intrcnts for the Hyper-V counters.
PR: 229429, 130483
Reviewed by: kib, royger, cem
Tested by: royger (Xen), kib (DMAR)
Approved by: re (gjb)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16861
netascii is obsolete and inefficient. It isn't even supported by many
clients. Better to use binary mode by default.
Reviewed by: cem
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D16869
The current kernel ifunc implementation creates a PLT entry for each
ifunc definition. ifunc calls therefore consist of a call to the
PLT entry followed by an indirect jump. The jump target is written
during boot when the kernel linker resolves R_[*]_IRELATIVE relocations.
This implementation is defined by requirements for userland code, where
text relocations are avoided. This requirement is not present for the
kernel, so the implementation has avoidable overhead (namely, an extra
indirect jump per call).
Address this for now by adding a special option to the static linker
to inhibit PLT creation for ifuncs. Instead, relocations to ifunc call
sites are passed through to the output file, so the kernel linker can
enumerate such call sites and apply PC-relative relocations directly
to the text section. Thus the overhead of an ifunc call becomes exactly
the same as that of an ordinary function call. This option is only for
use by the kernel and will not work for regular programs.
The final form of this optimization is up for debate; for now, this
change is simple and static enough to be acceptable as an interim
solution.
Reviewed by: emaste
Discussed with: arichardson, dim
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16748
Fixes courtesy of arichardson and jmg:
- HACKING was pointing to the wrong place
- Added headers were being relied on implicitly, but libstdc++ did not
comply with the unspoken wishes of dtc.
MFC after: 1 week
grep(1) changes:
- Pet mandoc & igor.
- Stylize the text more with macros when appropriate.
- Stylize equal signs in long options (e.g., "--color=auto") with
the "Cm" macro as suggested by mdoc(7).
- Add missing arguments to --exlude, --exclude-dir, --include and
--include-dir.
- Remove a duplicate entry for the --context flag.
- Use a list in the EXAMPLES sections to make it easier to tell
which paragraphs belong to which example.
- Cross reference zgrep(1).
zgrep(1) changes:
- Fix Nd.
- Split synopsis into paragraphs for readability.
- Cross reference bzip(1), grep(1) and xz(1).
Reviewed by: bcr
Approved by: mat (mentor)
Differential Revision: https://reviews.freebsd.org/D16779
It was #ifdef'd out in the 4.4BSD import and hasn't been re-enabled
since then.
Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16804
Instead of doing a second pass to skip empty lines if we've specified -I, go
ahead and check both at once. Ignore critera has been split out into its own
function to try and keep the logic cleaner.
As noted by cem in r338035, coccinelle invokes diff(1) with the -B flag.
This was not previously implemented here, so one was forced to create a link
for GNU diff to /usr/local/bin/diff
Implement the -B flag and add some primitive tests for it. It is implemented
in the same fashion that -I is implemented; each chunk's lines are scanned,
and if a non-blank line is encountered then the chunk will be output.
Otherwise, it's skipped.
MFC after: 2 weeks
Notable fixes:
- Overlays may now be generated properly without -@
- /__local_fixups__ were not including unit address in their structure
- The error reporting a magic token was misleading, reporting
"Bad magic token in header. Got d00dfeed expected 0xd00dfeed"
if the token was missing. This has been split out into a separate message.
MFC after: 1 week
The fix is only partial and causes an asymmetry which breaks a test in
multi_test.sh.
We should consider both parts of the issue found in OpenBSD[1], but for now
just revert the change.
[1] http://undeadly.org/cgi?action=article;sid=20180728110010
Reported by: asomers
The bug was that isalnum() is not exactly equivalent to previous code which
also allowed characters "$" and "_", so check for those explicitly.
Reported by: tuexen@
While STACKSIZE macro is indeed problematic on some systems, the commits
were wrong to shrink il[] and cstk[], because they need to be of the same
size as p_stack[] as they're accessed with the same index ps.tos.
to match reality (slightly different to what was submitted in the
PR: use english word instead of math-symbol).
- Wrap the corresponding part to below 80 characters per line.
Submitted by: yamagi@yamagi.org
PR: 202202
Sponsored by: Essen Hackathon
This helps with pkgbase as it tags this as a config file so it is handled as
such
Approved by: allanjude (mentor)
Sponsored by: Essen Hackathon
Differential Revision: https://reviews.freebsd.org/D16673
SVN r337458 erroneously partially reverted r265885.
This is immediately visible when running the Kyua/ATF tests for
usr.bin/printf, which actually test sh's printf builtin.
PR: 229641
When invoked on a large list of files, it is most common for a small number of
uids/gids to own most of the results.
Like ls(1), use pwcache(3) to avoid repeatedly looking up the same IDs.
Example microbenchmark and non-scientific results:
$ time (find /usr/src -type f -print0 | xargs -0 stat >/dev/null)
BEFORE:
3.62s user 5.23s system 102% cpu 8.655 total
3.47s user 5.38s system 102% cpu 8.647 total
AFTER:
1.23s user 1.81s system 108% cpu 2.810 total
1.43s user 1.54s system 107% cpu 2.754 total
Does this microbenchmark have any real-world significance? Until a use case
is demonstrated otherwise, I doubt it. Ordinarily I would be resistant to
optimizing pointless microbenchmarks in base utilities (e.g., recent totally
gratuitous changes to yes(1)). However, the pwcache(3) APIs actually
simplify stat(1) logic ever so slightly compared to the raw APIs they wrap,
so I think this is at worst harmless.
PR: 230491
Reported by: Thomas Hurst <tom AT hur.st>
Discussed with: gad@
Using a space as the magic character would result in problems if the command
started with a number:
- For a 'valid' number n, n < size of argv, it would erroneously get
replaced with that argument; e.g. `apply -a ' ' -d 1rm x => `execxrm x`
- For an 'invalid' number n, n >= size of argv, it would segfault.
e.g. `apply -a ' ' 2to3 test.py` would try to access argv[2]
This problem occurred because apply(1) would prepend "exec " to the command
string before doing the actual magic number replacements, so it would come
across "exec 2to3 1" and assume that the " 2" is also a magic number to be
replaced.
Re-work this to instead just append "exec " to the command sbuf and
workaround the ugliness. This also simplifies stuff in the process.
PR: 226948
Submitted by: Tobias Stoeckmann <tobias@stoeckmann.org>
MFC after: 1 week
The precision with the conversion specifier b is specified by POSIX: see
point 7 in the reference documentation.
This corrects previous wrong log in r337440.
Reference: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html
PR: 229641
Reported by: Rudolf Cejka
Submitted by: Garrett D'Amore (illumos)
MFC after: 1 week
