mkimg(1) uses a swap file to back input file chunks. When the output file
is being written out, blocks of the swap file are mapped and their contents
copied. This causes the backing VM pages to enter the active queue, and when
the output file is large relative to system memory (as is generally the
case), can result in a shortfall of inactive memory. This causes the
pagedaemon to aggressively scan the active queue and swap out process
memory in an attempt to meet the shortfall. Because mkimg's input files
are typically the intermediate result of some build process, there's no
need to push them all through the active queue. Use madvise(2) to indicate
that the backing pages may be reclaimed in preference to active pages. In
the case of the swap file, these pages will be freed as soon as mkimg
exits anyway.
When using mkimg on a desktop-class system with large amounts of dirty
process memory, this change substantially improves mkimg runtime and
reduces swap usage.
Reviewed by: marcel
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D6654
discovery without attaching to the targets ("iscsictl -Ad ... -e off"),
and then attach to selected ones ("iscsictl -Mi ... -e on").
PR: 204129
MFC after: 1 month
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6633
Somehow the /usr/include path got lost in this particular case.
Just pass it along from --sysroot as was already done for
DIRDEPS_BUILD.
Sponsored by: EMC / Isilon Storage Division
In addition to the previous change I made to ar.c, pull in another
basename() related fix. This change is similar to the one made to the
ELF Toolchain version of ar, with the difference that the ELF Toolchain
version lacks error handling for the strdup() call.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D6467
Summarizing the findings in the OpenBSD list:
This solves a reproduceable issue with very recent Mesa where REG_NOTBOL
combined with a match at the begin of the string causes our regex library
to treat the word as not begin of word.
Thanks to Martijn van Duren and Ingo Schwarze for taking the time to
solve this in the least invasive way.
PR: 209352, 209387
Taken from: openbsd-tech (Martijn van Duren)
MFC after: 1 month
st_mtim was being incorrectly described as "stime=", not "mtime=". This was
introduced with the original feature commit (r176471).
MFC after: 1 week
PR: 209699
Submitted by: naddy
Sponsored by: EMC / Isilon Storage Division
The density code and bits per mm values were obtained from an
actual drive density report.
The number of tracks were obtained from an LTO-7 hardware
announcement on IBM's web site.
Sponsored by: Spectra Logic
MFC after: 3 days
If basename() uses "char *", we shouldn't do the intermediate
assignment, as that field is of type "const char *". Simply call
basename() on the command line argument directly.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D6463
This should fix the build on older stable/10, since install is a bootstrap
tool.
Pending a decision how to fix this properly, revert utimensat usage. Copies
with the -p option will again appear older than the original almost always,
but -p is not commonly used.
expand(). Never return the name parameter, which could be a the buf[]
buffer which is allocated on the stack by getdeadletter() and which
would then be used after getdeadletter() has returned.
Reported by: Coverity
CID: 1199383
MFC after: 1 week
memcpy() instead. It's probably a bit more optimal in this case
anyway. [1]
The program logic leading up to the creation of the strncpy/memcpy
destination buffer is a bit hairy. Add a call to assert() to make
it clear what is happening here and detect any potential buffer
overruns in the future.
Check a couple syscall error returns. Ignore the EEXIST error from
link() to preserve existing behavior. [2] [3]
Reported by: Coverity
CID: 1009659 [1], 1009349 [2], 1009350 [3]
Update libarchive to 3.2.0
New features:
- new bsdcat command-line utility
- LZ4 compression (in src only via external utility from ports)
- Warc format support
- 'Raw' format writer
- Zip: Support archives >4GB, entries >4GB
- Zip: Support encrypting and decrypting entries
- Zip: Support experimental streaming extension
- Identify encrypted entries in several formats
- New --clear-nochange-flags option to bsdtar tries to remove noschg and
similar flags before deleting files
- New --ignore-zeros option to bsdtar to handle concatenated tar archives
- Use multi-threaded LZMA decompression if liblzma supports it
- Expose version info for libraries used by libarchive
Patched files (fixed compiler warnings):
contrib/libarchive/cat/bsdcat.c (vendor PR #702)
contrib/libarchive/cat/bsdcat.h (vendor PR #702)
contrib/libarchive/libarchive/archive_read_support_format_mtree.c (PR #701)
contrib/libarchive/libarchive_fe/err.c (vendor PR #703)
MFC after: 1 month
Relnotes: yes
to ensure that the destination is NUL terminated. Length truncation
of one more character should not be an issue since encoding values
that long are not supported by libc. The destination string is
treated as a NUL terminated string, but it is only passed to strcmp()
for comparison to a set of shorter, fixed length strings, so this
is not a serious problem.
Reported by: Coverity
CID: 974769
MFC after: 1 week
In the case where a file lacks a trailing newline, there is some "evil" code to
reverse goto the tokenizing code ("make_token") for the final token in the
file. In this case, 'fd' is closed more than once. Use a negative sentinel
value to guard close(2), preventing the double close.
Ideally, this code would be restructured to avoid this ugly construction.
Reported by: Coverity
CID: 1006123
Sponsored by: EMC / Isilon Storage Division
Close the fd the poll error was detected on, rather than the last opened fd, to
fix the double-close.
Use -1 to make it explict which int variables no longer own socket file
descriptors.
Actually shrink, rather than grow, the poll timeout to match comment.
Reported by: Coverity
CID: 1304860, 1305616
Sponsored by: EMC / Isilon Storage Division
This logic was added to the whois() function in r281959, but could easily be
its own routine. In this case, I think the abstraction makes both functions
easier to reason about.
This precedes some Coverity-suggested cleanup.
Sponsored by: EMC / Isilon Storage Division
Use size of destination buffer, rather than a constant that may or may not
correspond to the source buffer, to restrict the length of copied strings. In
particular, pr_fname has 16+1 characters but MAXCOMLEN is 18+1.
Use strlcpy instead of strncpy to ensure the result is nul-terminated. This
seems to be what is expected of these fields.
Reported by: Coverity
CIDs: 1011302, 1011378
Sponsored by: EMC / Isilon Storage Division
In the presence of the --diff-pid argument, it is possible for 'diffpipe' to be
NULL. Only fclose() it if it was initialized.
Reported by: Coverity
CID: 1355183
Sponsored by: EMC / Isilon Storage Division
When getline(3) in 2009 was added a _WITH_GETLINE guard has also been added.
This rename is made in preparation for the removal of this guard
Obtained from: NetBSD
Simplify redundant malloc'ing in sed -e.
It is causing havoc in the ports tree:
===> Configuring for wxsvg-1.5.7
sed: 1: "/gcc_dir=\\`/s/gcc /$CC /": bad flag in substitute command: '/'
*** Error code 1
===> Patching for vips-8.3.1
sed: 1: "1s|^#![[:space:]]*/usr/ ...": bad flag in substitute command: 's'
*** Error code 1
PR: 195929
Reported by: danilo
When encountering an -e argument, sed currently mallocs a string to COPY
the optarg -- with '\n' appended. The appendage does not seem necessary --
indeed, the same call to add_compunit processing the sole command (given
without -e) passes the *argv verbatim: without making a copy, and without
appending newline.
This matches what is done in other BSDs.
Submitted by: Mikhail T.
PR: 195929
MFC after: 2 weeks
Rewrite the main loop of the "sed s/..." command, shortening it by ten
lines and simplifying it by removing the switch statement implementing
/g, /1, and /2 separately and repetitively.
This will be needed to bring a fix from OpenBSD later.
Obtained from: OpenBSD (schwarze CVS Rev. 1.18)
MFC after: 3 weeks
after r298107
Summary of changes:
- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)
Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.
MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
There are a couple of places in the source three where we call
basename() on constant strings. This is bad, because the prototype
standardized by POSIX allows the implementation to use its argument as a
storage buffer.
This change eliminates some of these unportable calls to basename() in
cases where it was only added for cosmetical reasons, namely to trim
argv[0]. There's nothing wrong with setting argv[0] to the full path.
Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D6093
This contains only bug fixes, no new features. The repository format is
also unchanged from 1.9.2. Full list of changes between 1.9.4 and
earlier versions:
https://svn.apache.org/repos/asf/subversion/tags/1.9.4/CHANGES
Note that the two security issues fixed in 1.9.4 (CVE-2016-2167 and
CVE-2016-2168) do not affect the version of Subversion in the FreeBSD
base system, since neither SASL nor Apache modules are enabled.
Relnotes: yes
MFC after: 2 weeks
Import sdiff(1) from the diff version written by Raymond Lai,
improved during GSoC 2012 by Jesse Hagewood.
Compared to the version done in during that summer of code:
- Remove the zlib frontend: zsdiff
- Compatible output (column size and separators) with GNU sdiff
Compared to GNU sdiff in ports:
- The only difference is padding using spaces vs tabs
Compared to OpenBSD and NetBSD import:
- Implement missing options (including long options) from GNU sdiff
- Improved support for the edition mode (signal handling)
- Output visually compatible with GNU sdiff: size of columns
While here import regression tests from NetBSD adapted to fit the output as
expected by GNU sdiff
Reviewed by: emaste (in part)
Obtained from: OpenBSD, NetBSD, GSoC 2012
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D5981
Differential Revision: https://reviews.freebsd.org/D6032 (diff with NetBSD version)
Differential Revision: https://reviews.freebsd.org/D6033 (diff with OpenBSD version)
Fix a related typo while here.
Note, this change results in the Kyuafile inclusion in the runtime
package, which needs to be fixed, however addresses the PR as far
as I can tell in my tests.
PR: 209114
Submitted by: ngie
Sponsored by: The FreeBSD Foundation
o Split the compression across several worker threads. By default, "several"
matches number of CPUs, capped at 24 for sanity when running on a very big
hardwares. Provide option to set that number manually;
o Fix bug inherited from the mkulzma (R.I.P) which degraded already slow LZMA
compression even further by calling function to release compression state
after processing each block.
It is neither documented as required nor actually required by the LZMA
library. This caused spree of system calls to release memory and then map
it again for every block. LZMA compression is more than 2x faster after this
change alone;
o Record time it takes to do compression and report throughput achieved.
o Add simple first-level 256 entry hash table for de-dup code, so it's not
becoming a bottleneck at big files.
It's provided by sys.mk so there's no need to derive it from ${.CURDIR}.
Suggested by: ngie
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5998
- notionally support a 'history file' flag. This doesn't do much now,
but is there to prevent scripts written against GNU units from
breaking
- correctly gracefully quit rather than exit (this will make it easier
to support a history file in the future)
- remove the "t" flag from fopen which was there to support windows. We
have not supported windows since at the latest, the introduction of
capsicum.
