fixes and quality of life improvements.
While there are security issues in this time frame that affect usage as a
server (eg: linked into apache), this isn't possible here.
in units(1). The most visible is the removal of libedit warnings
about being unable to open termcap database.
Reviewed by: eadler@
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3322
The localedef tool can read entire (and unmodified) CLDR posix definition
files, and generate all 6 LC categories: LC_COLLATE, LC_CTYPE, LC_TIME,
LC_NUMERIC, LC_MONETARY and LC_MESSAGES.
This tool has a long history with Solaris. The Nexenta developers
modified it to read CLDR files and created the much richer collation
formats. The libc collation functions have to be modified to read the
new format (called "BSD-1.0") and to handle the new data structures.
The result will be that locale-sensitive tools and functions will now
properly sort multibyte and unicode strings.
Obtained from: Dragonfly
is taken to match the geometry and only when the geometry is max'd
out, is the actual recorded size taken.
Note that qemu has the same logic for the fixed VHD format. However
that is known to conflict with Microsoft Azure, where the recorded
size of the image is what counts.
Pointed out by: gjb@
especially useful now that libc's open() always calls openat(). While here,
fix a few other things:
- Decode the mode argument passed to access(), eaccess(), and faccessat().
- Decode the atfd paramete to pretty-print AT_FDCWD.
- Decode the special AT_* flags used with some of the *at() system calls.
- Decode arguments for fchmod(), lchmod(), fchown(), lchown(), eaccess(),
and futimens().
- Decode both of the timeval structures passed to futimes() instead of just
the first one.
tightening sanity check of the input. [1]
While I'm there also replace ed(1) with red(1) because we do
not need the unrestricted functionality. [2]
Obtained from: Bitrig [1], DragonFly [2]
Security: CVE-2015-1418 [1]
If the resulting argument is longer than MAXPATHLEN, realloc() was called to
extend the space, but the new pointer was not correctly stored.
Different from what OpenBSD has done, rewrite brace_subst() to calculate the
necessary space first and realloc() at most once.
As before, the e_len fields are not updated in case of a realloc.
Therefore, a following long argument will do another realloc.
PR: 201750
MFC after: 1 week
length. In particular, instead of blinding fetching 1k blocks, do an initial
fetch up to the end of the current page followed by page-sized fetches up to
the maximum size. Previously if the 1k buffer crossed a page boundary and
the second page was not valid, the entire operation would fail.
Revert r286102 and apply a cleaner fix.
Tested for overflows by FORTIFY_SOURCE GSoC (with clang).
Suggested by: bde
Reviewed by: Oliver Pinter
Tested by: Oliver Pinter
MFC after: 3 days
ELF Tool Chain elfcopy is nearly a drop-in replacement for GNU objcopy,
but does not currently support PE output which is needed for building
x86 UEFI bits.
Add a src.conf knob to allow installing it as objcopy and set it by
default for aarch64 only, where we don't have a native binutils.
Reviewed by: bapt
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2887
Ar cannot handle UIDs with more than 6 digits, and storing the mtime,
uid, gid and mode provides little to negative value anyhow for ar's
uses. Turn on deterministic (-D) mode by default; it can be disabled by
the user with -U.
PR: 196929
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3190
making clear transits between different states, and avoids bug with
handling repeated $'s.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D3221
Rationale: ident(1) is useful out of RCS, lot of scripts are using ident(1) and
failing when base is built WITHOUT_RCS.
This version is:
- fully compatible with RCS 5.7 ident.
- fully compatible with RCS 5.9 ident.
- passes all ident test from GNU RCS 5.9 test suite
This version has support for: svn extension for the Keyword id (double colon and
# before last $)
Différences with GNU RCS ident:
- no long options as found in GNU RCS 5.9 (but not commented there).
- '-V' reports nothing but has been added for compatibility.
Differential Revision: https://reviews.freebsd.org/D3200
Reviewed by: pfg
This is required by our FORTIFY_SOURCE implementation as it
does more inlining. As a rule of thumb, FORTIFY_SOURCE doubles
the number of inlines except that in grep inlining
blows up for some reason.
This is required in order for us to support deterministic mode by
default. If multiple -D or -U options are specified on the command
line, the final one takes precedence. GNU ar also uses -U for this.
An equivalent change will be applied to ELF Tool Chain's version of ar.
PR: 196929
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3175
If apropos(1) and whatis(1) are not hardlinks to man(1) that means the system is
using mandocdb, then man -k should spawn apropos(1) and/or whatis(1) directly
Reported by: kevlo
Tested by: kevlo
Sponsored by: gandi.net
In the kernel, structs such as tcpstat are manipulated as an array of
counter_u64_t (uint64_t *), but made visible to userland as an array of
uint64_t. kread_counters() was previously copying the counter array into
user space and sequentially overwriting each counter with its value. This
mostly affects IPsec counters, as other counters are exported via sysctl.
PR: 201700
Tested by: Jason Unovitch
MFC after: 1 week
According to the last(1) man page, the "reboot" pseudo-user should print
all system reboot entries. This got broken by the utmpx import, as
records are typed.
Re-add support for "last reboot" by specifically matching against
SHUTDOWN_TIME and BOOT_TIME records.
PR: 168844
Submitted by: matthew@
MFC after: 1 month
This is based on work done by jeff@ and jhb@, as well as the numa.diff
patch that has been circulating when someone asks for first-touch NUMA
on -10 or -11.
* Introduce a simple set of VM policy and iterator types.
* tie the policy types into the vm_phys path for now, mirroring how
the initial first-touch allocation work was enabled.
* add syscalls to control changing thread and process defaults.
* add a global NUMA VM domain policy.
* implement a simple cascade policy order - if a thread policy exists, use it;
if a process policy exists, use it; use the default policy.
* processes inherit policies from their parent processes, threads inherit
policies from their parent threads.
* add a simple tool (numactl) to query and modify default thread/process
policities.
* add documentation for the new syscalls, for numa and for numactl.
* re-enable first touch NUMA again by default, as now policies can be
set in a variety of methods.
This is only relevant for very specific workloads.
This doesn't pretend to be a final NUMA solution.
The previous defaults in -HEAD (with MAXMEMDOM set) can be achieved by
'sysctl vm.default_policy=rr'.
This is only relevant if MAXMEMDOM is set to something other than 1.
Ie, if you're using GENERIC or a modified kernel with non-NUMA, then
this is a glorified no-op for you.
Thank you to Norse Corp for giving me access to rather large
(for FreeBSD!) NUMA machines in order to develop and verify this.
Thank you to Dell for providing me with dual socket sandybridge
and westmere v3 hardware to do NUMA development with.
Thank you to Scott Long at Netflix for providing me with access
to the two-socket, four-domain haswell v3 hardware.
Thank you to Peter Holm for running the stress testing suite
against the NUMA branch during various stages of development!
Tested:
* MIPS (regression testing; non-NUMA)
* i386 (regression testing; non-NUMA GENERIC)
* amd64 (regression testing; non-NUMA GENERIC)
* westmere, 2 socket (thankyou norse!)
* sandy bridge, 2 socket (thankyou dell!)
* ivy bridge, 2 socket (thankyou norse!)
* westmere-EX, 4 socket / 1TB RAM (thankyou norse!)
* haswell, 2 socket (thankyou norse!)
* haswell v3, 2 socket (thankyou dell)
* haswell v3, 2x18 core (thankyou scott long / netflix!)
* Peter Holm ran a stress test suite on this work and found one
issue, but has not been able to verify it (it doesn't look NUMA
related, and he only saw it once over many testing runs.)
* I've tested bhyve instances running in fixed NUMA domains and cpusets;
all seems to work correctly.
Verified:
* intel-pcm - pcm-numa.x and pcm-memory.x, whilst selecting different
NUMA policies for processes under test.
Review:
This was reviewed through phabricator (https://reviews.freebsd.org/D2559)
as well as privately and via emails to freebsd-arch@. The git history
with specific attributes is available at https://github.com/erikarn/freebsd/
in the NUMA branch (https://github.com/erikarn/freebsd/compare/local/adrian_numa_policy).
This has been reviewed by a number of people (stas, rpaulo, kib, ngie,
wblock) but not achieved a clear consensus. My hope is that with further
exposure and testing more functionality can be implemented and evaluated.
Notes:
* The VM doesn't handle unbalanced domains very well, and if you have an overly
unbalanced memory setup whilst under high memory pressure, VM page allocation
may fail leading to a kernel panic. This was a problem in the past, but it's
much more easily triggered now with these tools.
* This work only controls the path through vm_phys; it doesn't yet strongly/predictably
affect contigmalloc, KVA placement, UMA, etc. So, driver placement of memory
isn't really guaranteed in any way. That's next on my plate.
Sponsored by: Norse Corp, Inc.; Dell
for a long time, quota(1) utility supported only PF_INET.
- Clean up confusing changes in f_mntfromname.
- Add an entry for rquotad with rpc/udp6 to inetd.conf.
PR: 194084
Notable upstream commits (upstream revision in parens):
- Add a JSON producer to LLDB (228636)
- Don't crash on bad DWARF expression (228729)
- Add support of DWARFv3 DW_OP_form_tls_address (231342)
- Assembly profiler for MIPS64 (232619)
- Handle FreeBSD/arm64 core files (233273)
- Read/Write register for MIPS64 (233685)
- Rework LLDB system initialization (233758)
- SysV ABI for aarch64 (236098)
- MIPS software single stepping (236696)
- FreeBSD/arm live debugging support (237303)
- Assembly profiler for mips32 (237420)
- Parse function name from DWARF DW_AT_abstract_origin (238307)
- Improve LLDB prompt handling (238313)
- Add real time signals support to FreeBSDSignals (238316)
- Fix race in IOHandlerProcessSTDIO (238423)
- MIPS64 Branch instruction emulation for SW single stepping (238820)
- Improve OSType initialization in elf object file's arch_spec (239148)
- Emulation of MIPS64 floating-point branch instructions (239996)
- ABI Plugin for MIPS32 (239997)
- ABI Plugin for MIPS64 (240123)
- MIPS32 branch emulation and single stepping (240373)
- Improve instruction emulation based stack unwinding on ARM (240533)
- Add branch emulation to aarch64 instruction emulator (240769)
Update setkey and libipsec to understand aes-gcm-16 as an
encryption method.
A partial commit of the work in review D2936.
Submitted by: eri
Reviewed by: jmg
MFC after: 2 weeks
Sponsored by: Rubicon Communications (Netgate)
This hack is very fragile, and was broken on powerpc64 when metamode was
introduced. Removing it survives a buildworld for all architectures, and
fixes the build on powerpc64.
Fault in the buffer prior to writing to workaround poor performance due
to interaction with kernel fs deadlock avoidance code. See the comment
prior to vn_io_fault_doio() in sys/kern/vfs_vnops.c for details of the
issue.
On my stable/10 desktop with a 16MB obj.o and "ar r out.a obj.o" I see
the following run times (seconds):
x ar.r284891
+ ar.patched
+----------------------------------------------------------------------+
|+ |
|+ x|
|+ xx|
|A |A|
+----------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 1.307 1.321 1.315 1.3143333 0.0070237692
+ 3 0.020 0.023 0.022 0.021666667 0.0015275252
Difference at 95.0% confidence
-1.29267 +/- 0.0115203
-98.3515% +/- 0.876513%
(Student's t, pooled s = 0.00508265)
Thanks to kib for diagnosing and explaining the issue and suggesting
the workaround.
Reviewed by: eadler, kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2933
The code path to support units conversions from the command line
need not initialize neither libedit nor the history. Therefore, only do
that when in interactive mode.
This hides the issue reported in PR bin/201167 whereby running commands
of the form 'echo "$(units ft in)"' would corrupt the terminal. The real
issue causing the corruption most likely still remains somewhere.
PR: bin/201167
Differential Revision: D2935
Reviewed by: eadler
When requesting install(1) to only make relative symlinks, by pass all the
done to actually compute the relative symlink if the path given in argument is
already a relative path
The image is not accepted for provisioning otherwise. Bump the
VHD creator tool version and the version of mkimg to signify our
success in provisioning.
Note that this also imapcts the dynamic VHD images.
Tested by: gjb@
of megabytes. This is on top of having the image rounded to the
matching geometry of the image size.
By rounding up to the next MB after rounding to the geometry, we
lost idempotency. Subsequent calls to resize the image will keep
increasing the image size.
Tested by: gjb@
really need it can find it in the devel/fmake port or pkg install fmake.
Note: This commit is orthogonal to the question 'can we fmake buildworld'.
Differential Revision: https://reviews.freebsd.org/D2840
address is loopback. So it is shown if both are not loopback.
The man page says that it is shown if the local or foreign
address is not loopback. Change the man page to reflect the
code.
MFC after: 3 days
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
multiple of the cylinder size. This is what qemu-img seems to
be doing. Make sure to handle boundary cases where increasing
the image size by 1 cyclinder's worth would also result in a
change of geometry.
contrib/top/top.local.H to contrib/top/top.local.hs.
This fixes a build breakage when src is on a case-
insensitive file system -- we never properly create
top.x nor top.local.h. Change the makefile accordingly.
MFC after: 3 days
This change among other things improve search capabilities over the manpages
allowing fine grain query.
A new build option WITHOUT_MANDOCDB has been added to keep the ancient version
of the database and the tools. The plan is to entirely remove this option before
11.0-RELEASE.
Differential Revision: https://reviews.freebsd.org/D2603
examined via 'vmstat -o'. It can be used to determine which files are
using physical pages of memory and how much each is using.
Differential Revision: https://reviews.freebsd.org/D2277
Reviewed by: alc, kib
MFC after: 2 weeks
Sponsored by: Norse Corp, Inc. (forward porting to HEAD/10)
the work to the main thread... This fixes a possible crash if SIGINFO
is delivered at the wrong time...
This still leaves getrusage broken for some reason, but I believe that
is a kernel issue and cannot be fixed here...
needs to be enabled by adding "kern.racct.enable=1" to /boot/loader.conf.
Differential Revision: https://reviews.freebsd.org/D2407
Reviewed by: emaste@, wblock@
MFC after: 1 month
Relnotes: yes
Sponsored by: The FreeBSD Foundation
This extends the existing support for -NORID handles
to include -NICAT, -ARIN, and -RIPE handles.
The suffix machinery is also used to work around a problem with
lack of referrals from the Nominet server for .uk: names under
.ac.uk need to be queried directly at JANET's whois server.
Document the RA_SERVER environment variable.
Remove the obsolete -R option, and discourage people from trying
to use the Network Solutions server.
Describe the default behaviour in the DESCRIPTION section,
not buried in the options.
and BSD-style escape-control-char sequences in the input stream.
Submitted by: schwarze at OpenBSD
Discussed with: schwarze at OpenBSD
Obtained from: OpenBSD
o Restore historical behaviour of appending '*' if interface is down,
and we have enough space to print it (usually we don't). [1]
o Do not truncate interface names when printing in encoded format.
o Report interface flags into encoded format.
PR: 199873 [1]
Sponsored by: Nginx, Inc.
Makes col(1) respect POSIX again for escape sequences as decribed in its manpage
The bug was introduced in CSRG in 1990
This also reverts r280911
Differential Revision: https://reviews.freebsd.org/D2424
Reviewed by: jhb
MFC after: 2 weeks
first entry reported by the relative mode (-R).
o Properly print negative offsets, which I guess may happen if
records get re-ordered somehow, possibly due to the locking. Right
now we report huge bogus diff (i.e. 2 seconds or so).
Attempt to connect to alternate addresses if the connect doesn't
succeed in 180ms, and cut wait time between connection attempts
in half for each additional, if no connection was established.
Take the first connection to succeed and close the others.
This makes whois more responsive when e.g. IPv6 service took long
time or fail to respond.
PR: 158125
Submitted by: Mark Andrews <marka isc org> (with changes from me)
MFC after: 2 weeks
