This is a painful change, but it is needed. On the one hand, we avoid
modifying them, and this slows down some ideas, on the other hand we still
eventually modify them and tools like netstat(1) never work on next version of
FreeBSD. We maintain a ton of spares in them, and we already got some ifdef
hell at the end of tcpcb.
Details:
- Hide struct inpcb, struct tcpcb under _KERNEL || _WANT_FOO.
- Make struct xinpcb, struct xtcpcb pure API structures, not including
kernel structures inpcb and tcpcb inside. Export into these structures
the fields from inpcb and tcpcb that are known to be used, and put there
a ton of spare space.
- Make kernel and userland utilities compilable after these changes.
- Bump __FreeBSD_version.
Reviewed by: rrs, gnn
Differential Revision: D10018
Decode the last argument to ioctl() as a pointer rather than an int.
Eventually this could use 'int' for the _IOWINT() case and pointers for
all others.
The last argument to sendto() is a socklen_t value, not a pointer.
Provides:
amount of compressed data
logical size of compressed data (how much it would have taken uncompressed)
compression ratio (logical size : total ARC size)
Overhead (space consumed for compression headers)
Example output:
ARC: 31G Total, 18G MFU, 9067M MRU, 2236K Anon, 615M Header, 2947M Other
25G Compressed, 54G Uncompressed, 1.76:1 Ratio, 2265M Overhead
Reviewed by: jpaetzel, smh, imp, jhb (previous version)
MFC after: 2 week
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D9829
Previously, the offset in a system call description specified the
array index of the start of a system call argument. For most system
call arguments this was the same as the index of the argument in the
function signature. 64-bit arguments (off_t and id_t values) passed
on 32-bit platforms use two slots in the array however. This was
handled by adding (QUAD_SLOTS - 1) to the slot indicies of any
subsequent arguments after a 64-bit argument (though written as ("{
Quad, 1 }, { Int, 1 + QUAD_SLOTS }" rather than "{ Quad, 1 }, { Int, 2
+ QUAD_SLOTS - 1 }"). If a system call contained multiple 64-bit
arguments (such as posix_fadvise()), then additional arguments would
need to use 'QUAD_SLOTS * 2' but remember to subtract 2 from the
initial number, etc. In addition, 32-bit powerpc requires 64-bit
arguments to be 64-bit aligned, so if the effective index in the array
of a 64-bit argument is odd, it needs QUAD_ALIGN added to the current
and any subsequent slots. However, if the effective index in the
array of a 64-bit argument was even, QUAD_ALIGN was omitted.
This approach was messy and error prone. This commit replaces it with
automated pre-processing of the system call table to do fixups for
64-bit argument offsets. The offset in a system call description now
indicates the index of an argument in the associated function call's
signature. A fixup function is run against each decoded system call
description during startup on 32-bit platforms. The fixup function
maintains an 'offset' value which holds an offset to be added to each
remaining system call argument's index. Initially offset is 0. When
a 64-bit system call argument is encountered, the offset is first
aligned to a 64-bit boundary (only on powerpc) and then incremented to
account for the second argument slot used by the argument. This
modified 'offset' is then applied to any remaining arguments. This
approach does require a few things that were not previously required:
1) Each system call description must now list arguments in ascending
order (existing ones all do) without using duplicate slots in the
register array. A new assert() should catch any future
descriptions which violate this rule.
2) A system call description is still permitted to omit arguments
(though none currently do), but if the call accepts 64-bit
arguments those cannot be omitted or incorrect results will be
displated on 32-bit systems.
Tested on: amd64 and i386
.../usr.bin/diff/diff_test
Some minor adjustment needed to be done for :same as it currently
has the test script hardcoded into the test, instead of using an
idiom like $(dirname $0)
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Prefer ${SRCTOP}/foo over ${.CURDIR}/../../foo and ${SRCTOP}/usr.bin/foo
over ${.CURDIR}/../foo for paths in Makefiles.
Differential Revision: https://reviews.freebsd.org/D9932
Sponsored by: Netflix
Silence on: arch@ (twice)
longer a special case.
- Prefer PREFIX/share/man over PREFIX/man.
- Add /usr/local/share/man to man_default_path.
- Update manpath man page.
Reviewed by: bapt
The additional testcases use absolute paths for sources and targets,
as the other testcase which tested `-l sr` used flat relative paths in
the same directory.
Please note that these testcases do not test `-l a` -- that's already
addressed in the battery of tests.
MFC after: 1 week
Sponsored by: Dell EMC Isilon
Unfortunately kyua does not omit the path mismatch on failure, so it must be coded
into the error message.
Cache the values, run the test(1) call, then print out the values in an atf_fail
call to emit the required diagnostics to debug why things are failing.
MFC after: 1 week
Sponsored by: Dell EMC Isilon
The MANPATH environment variable behaviour is documented properly in the manpage
and it now has extended to new feature that allows to make MANPATH env variable
extending the default search path rather than overwriting it making the warning
painful
Reported by: kargl
MFC after: 1 week
mandoc.
If MANPATH begins with a colon, it is appended to the default list; if it ends
with a colon, it is prepended to the default list; or if it contains two
adjacent colons, the standard search path is inserted between the colons. If
none of these conditions are met, it overrides the standard search path.
Import the MANPATH description from mandoc into the man(1) man page
Reported by: kargl
MFC after: 1 week
localbase is not consistent with base for manpages:
/usr/local/man vs /usr/share/man adding share/man allows to fix that
inconsistency and would permit to remove tons of patches/modifications in the
ports tree
Some of the modifications from the previous summer of code has been integrated
Modification for compatibility with GNU diff output has been added
Main difference with OpenBSD:
Implement multiple GNU diff options:
* --ignore-file-name-case
* --no-ignore-file-name-case
* --normal
* --tabsize
* --strip-trailing-cr
Make diff -p compatible with GNU diff
Implement diff -l
Make diff -r compatible with GNU diff
Capsicumize diffing 2 regular files
Add a simple test suite
Approved by: AsiaBSDcon devsummit
Obtained from: OpenBSD, GSoC
Relnotes: yes
The localdef(1) changes are breaking world:
00:18:40.750 /usr/src/share/colldef/af_ZA.UTF-8.src: 2421: error: Bad file
descriptor
I will fix them offline.
Reported by: lwshu and many others
Also some small cleanups to match better current illumos.
CID: 1338540, 1338541, 1338557, 1338566
Obtained from: illumos
Discussed with: Yuri Pankov (@Nexenta)
MFC after: 5 days
As discussed during AsiaBSDcon devsummit, import the manpage from OpenBSD which
is has been rewritten in mdoc(7) format making it readable by default with
mandoc, it also has been extended by OpenBSD to cover all awk(1) options
Obtained from: OpenBSD
MFH: 1 week
- Use OBJTOP/SRCTOP-relative paths when looking for include files and
strfile.
- Add FORTUNES_OBJ and FORTUNES_SRC to abbreviate usr.bin/fortune
pathing.
This is being done to simplify make output/idioms.
MFC after: 1 week
Reviewed by: bdrewery
Sponsored by: Dell EMC Isilon
Differential Revision: D9916
In most cases strfile is built as part of build-tools, but in the event that someone
cd'ed to the directory, tried to build from scratch, and had MK_GAMES=no previously,
the build would fail in .../datfiles , trying to find strfile .
Mark this directory tree "SUBDIR_PARALLEL" safe to help facilitate this, instead of
shuffling around the SUBDIR entries (all of the other Makefiles will build standalone).
MFC after: 1 week
Sponsored by: Dell EMC Isilon
reallocarray(3) is a non portable extension from OpenBSD. Given that it is
already in FreeBSD, make easier future merges by adopting in some cases
where the code has some shared heritage with OpenBSD.
Obtained from: OpenBSD
by usually being double-spaced due to auto-wrap at column 80.
r212771 increased width of the hostname field from 12 to 25. This was
supposed to allow for 80-column output with all 3 load averages taking
5 characters each, but it actually gave width exactly 80 and thus worse
than useless auto-wrap in that case. 3 wide load average fields are
unusual, but later expansion of another field gave the auto-wrap with
just 2 wide load average fields.
Change to dynamic field widths for all fields except the uptime. This
also fixes the formatting of high (above 9999) user counts and not
very high (above 9.99) load averages. The formatting for numbers now
breaks at 99999.99, but scientific notation should be used starting
well below that.
The field width for the uptime remains hard-coded to work consistently
for uptimes less than 10000 days, but this gives too much space for
small uptimes. Punctuation between fields could be improved in many
ways, for example by removing it.
test suite
This change does the following:
- Introduces symmetry in the test inputs/outputs by adding the exit
code to the files. This simplified the test driver notably by
requiring less filename/test name manipulation.
- Adds a test driver for the testcases added in r313544, patterned
after bin/sh/tests/functional_test.sh . The driver calls indent as
noted in r313544, with an exception: The $FreeBSD$ RCS keyword's
expansion is reindented with indent, which means that the output
differs from the expected output. Thus, all lines with $FreeBSD$
in them are deleted on the fly, both in the input file and the
output file.
The test inputs/outputs are copied to the kyua sandbox before the
test is run as the pathing in some of the files relies on pathing
normalized to the current directory (copying the files is the
easiest way to resolve the issue).
Approved by: pstef (maintainer)
Reviewed by: pstef
X-MFC with: r313544
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D9682
For linux the mmap offset must also be page aligned, and we
need to disable macros like __FBSDID()
Change the linux osdep_uuidgen() to use more portable gettimeofday().
Reviewed by: marcel
Update libarchive to version 3.3.1 (and sync with latest vendor dist)
Notable vendor changes:
PR #501: improvements in ACL path handling
PR #724: fix hang when reading malformed cpio files
PR #864: fix out of bounds read with malformed GNU tar archives
Documentation, style, test suite improvements and typo fixes.
New options to bsdtar that enable or disable reading and/or writing of:
Access Control Lists (--acls, --no-acls)
Extended file flags (--fflags, --no-fflags)
Extended attributes (--xattrs, --no-xattrs)
Mac OS X metadata (Mac OS X only) (--mac-metadata, --no-mac-metadata)
MFC after: 2 weeks
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
Prefer setvbuf() to setlinebuf() for portability.
Some style(9) and redundant tests for NULL.
These are only meant to ease up merging newer changes but we are skipping
changes done in order to accomodate OpenBSD's pledge support.
Obtained from: OpenBSD
MFC after: 2 weeks
r275234 addressed sort automatically converting 8-bit locales to UTF-8
by using "LANG=C sort", but LC_ALL overrides LANG if set, so the issue
may still be present depending on the user's environment. Use LC_ALL=C
instead.
Reported by: tests.reproducible-builds.org
Reviewed by: bapt
MFC after: 1 week
Sponsored by: The Linux Foundation / Core Infrastructure Initiative
Differential Revision: https://reviews.freebsd.org/D9765
The actual issue was the fact that if - was used then some restriction were
already set to stdin when we were applying caph_limit_stdio which was failing
due to the fact the fd was the fd was already restricted to lower rights.
Restricting stdio before actually opening the files prevent trying to raise the
right and fixes the issue.
And this allows to keep failing the program if restriction failed
Approved by: allanjude
Differential Revision: https://reviews.freebsd.org/D9723
When fed from a pipe, lam(1) would sometimes fail:
lam: unable to limit stdio: Capabilities insufficient
fixed regression in portsnap(8) introduced in r313938
This broke portsnap(8), the app that the capsicumization of lam(1) was
meant to secure.
# portsnap fetch update
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from ec2-eu-west-1.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Tue Feb 21 16:05:39 MSK 2017 to Tue Feb 21 16:59:30 MSK 2017.
Fetching 5 metadata patches.lam: unable to limit stdio: Capabilities insufficient
done.
Applying metadata patches... done.
Fetching 5 metadata files... lam: unable to limit stdio: Capabilities insufficient
/usr/sbin/portsnap: cannot open 8c94d2c3f8fcea20eb1fd82021566c99c63a010e6b3702ee11e7a491795bcfb8.gz: No such file or directory
metadata is corrupt.
Reported by: Vladimir Zakharov <zakharov.vv@gmail.com>, Ben Woods <woodsb02@gmail.com>
MSDOS and Windows GNU grep uses -u to mean "print byte offsets as if
running on an UNIX system." The option has no effect on systems that
do not use CRLF line endings.
PR: 171200
Submitted by: deeptech71@gmail.com, Anders Jensen-Waud
MFC after: 1 month
Rework part of the loop in grep_fgetln to return the rest of the line
and ensure that we still advance the buffer by the length of the rest
of the line.
PR: 165471
Submitted by: Kyle Evans <kevans91@ksu.edu>
MFC after: 1 month
lam(1) is used in portsnap(8), so lock it down
Reviewed by: emaste, cem, jonathan
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D8076
* Initialize correct parent in binary_operator's constructor.
* Include <errno.h> explicitly, otherwise errno is undefined (without
NDEBUG, this is accidentally 'fixed' by including <iostream>).
Reported by: matteo
MFC after: 3 days
for maketab.c
The former simplifies pathing in make/displayed output, whereas the latter was just
unnecessarily superfluous since .PATH referenced the path to maketab.c earlier on in
the Makefile.
MFC after: 1 week
Sponsored by: Dell EMC Isilon
These examples show expected behavior of indent(1). They are meant to be used
together with a regression test mechanism, either Kyua, a Makefile or perhaps
something else. The mechanism should in essence do this:
indent -P${test}.pro < ${test}.0 > ${test}.0.run
and compare ${test}.0.stdout to ${test}.0.run. If the files differ or the exit
status isn't 0, the test failed.
* ${test}.pro is an indent(1) profile: a list of options passed through a file.
The program doesn't complain if the file doesn't exist.
* ${test}.0 is a C source file which acts as input for indent(1). It doesn't
have to have any particular formatting, since it's the output that matters.
* ${test}.0.stdout contains expected output. It doesn't have to be formatted in
Kernel Normal Form as the point of the tests is to check for regressions in
the program and not to check that it always produces KNF.
Reviewed by: ngie
Approved by: pfg (mentor)
Differential Revision: https://reviews.freebsd.org/D9007
In the kernel, cache the machine and flags fields from ELF header to use in
the ELF header of a core dump. For gcore, the copy these fields over from
the ELF header in the binary.
This matters for platforms which encode ABI information in the flags field
(such as o32 vs n32 on MIPS).
Reviewed by: kib
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D9392
The S_ISREG check was restored, such that the code will again fail with
in-place replacements on symlinks
MFC after: 12 days
X-MFC with: r313277
Sponsored by: Dell EMC Isilon
Small summary
-------------
o Almost all IPsec releated code was moved into sys/netipsec.
o New kernel modules added: ipsec.ko and tcpmd5.ko. New kernel
option IPSEC_SUPPORT added. It enables support for loading
and unloading of ipsec.ko and tcpmd5.ko kernel modules.
o IPSEC_NAT_T option was removed. Now NAT-T support is enabled by
default. The UDP_ENCAP_ESPINUDP_NON_IKE encapsulation type
support was removed. Added TCP/UDP checksum handling for
inbound packets that were decapsulated by transport mode SAs.
setkey(8) modified to show run-time NAT-T configuration of SA.
o New network pseudo interface if_ipsec(4) added. For now it is
build as part of ipsec.ko module (or with IPSEC kernel).
It implements IPsec virtual tunnels to create route-based VPNs.
o The network stack now invokes IPsec functions using special
methods. The only one header file <netipsec/ipsec_support.h>
should be included to declare all the needed things to work
with IPsec.
o All IPsec protocols handlers (ESP/AH/IPCOMP protosw) were removed.
Now these protocols are handled directly via IPsec methods.
o TCP_SIGNATURE support was reworked to be more close to RFC.
o PF_KEY SADB was reworked:
- now all security associations stored in the single SPI namespace,
and all SAs MUST have unique SPI.
- several hash tables added to speed up lookups in SADB.
- SADB now uses rmlock to protect access, and concurrent threads
can do SA lookups in the same time.
- many PF_KEY message handlers were reworked to reflect changes
in SADB.
- SADB_UPDATE message was extended to support new PF_KEY headers:
SADB_X_EXT_NEW_ADDRESS_SRC and SADB_X_EXT_NEW_ADDRESS_DST. They
can be used by IKE daemon to change SA addresses.
o ipsecrequest and secpolicy structures were cardinally changed to
avoid locking protection for ipsecrequest. Now we support
only limited number (4) of bundled SAs, but they are supported
for both INET and INET6.
o INPCB security policy cache was introduced. Each PCB now caches
used security policies to avoid SP lookup for each packet.
o For inbound security policies added the mode, when the kernel does
check for full history of applied IPsec transforms.
o References counting rules for security policies and security
associations were changed. The proper SA locking added into xform
code.
o xform code was also changed. Now it is possible to unregister xforms.
tdb_xxx structures were changed and renamed to reflect changes in
SADB/SPDB, and changed rules for locking and refcounting.
Reviewed by: gnn, wblock
Obtained from: Yandex LLC
Relnotes: yes
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D9352
The use of DES for anything is discouraged, especially with a static IV of 0
If you still need bdes(1) to decrypt Kirk's video lectures, see
security/bdes in ports.
This commit brought to you by the FOSDEM DevSummit and the
"remove unneeded dependancies on openssl in base" working group
Reviewed by: bapt, brnrd
Relnotes: yes
Sponsored by: FOSDEM DevSummit
Differential Revision: https://reviews.freebsd.org/D9424
Use S_ISREG instead of manual & (also it's better to compare the
result from & and the pattern instead of just assuming it's one bit
value).
Pointed out by Tianjie Mao <tjmao tjmao com>.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D4827
Worker threads now use a pthread_cond_t to wait for work instead of
burning the cpu up.
Obtained from: DragonflyBSD (07774aea0ccf64a48fcfad8899e3bf7c8f18277a)
MFC after: 2 weeks
Note that mandoc does not use anymore sqlite3 but a home made database format
An important improvement has been made as well in makewhatis performance:
Tests on my laptop shows makewhatis on the entire system goes from 26s to 12s
result from & and the pattern instead of just assuming it's one bit
value).
Pointed out by Tianjie Mao <tjmao tjmao com>.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D4827
Avoid "unused variable 'i'" warnings in generated .c files by only
emitting the "int i;" for non-opaque arrays. Opaque arrays use
xdr_opaque() rather than iterating over the array.
Obtained from: OpenBSD (CVS rev 1.28)
MFC after: 1 week
when fortune and other games moved from /usr/games to /usr/bin; I am
removing rather than correcting it since we normally do not mention in
the FILES section the paths to programs in /usr/bin/.
PR: 215962
Reported by: Andras Farkas
* Rewrite r_buf to use standard tail queues instead of a hand-rolled
circular linked list. Free dynamic allocations when done.
* Remove an optimization for the case where the file is a multiple of 128KB
in size and there is a scarcity of memory.
* Add ATF tests for "tail -r" and its variants.
Reported by: Valgrind
Reviewed by: ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D9067
Instead of collecting statistics for each combination of ports and logical
units, that consumed ~45KB per LU with present number of ports, collect
separate statistics for every port and every logical unit separately, that
consume only 176 bytes per each single LU/port. This reduces struct
ctl_lun size down to just 6KB.
Also new IOCTL API/ABI does not hardcode number of LUs/ports, and should
allow handling of very large quantities.
MFC after: 2 weeks (probably keeping old API enabled for some time)
Load kvm symbols earlier to prevent rstat: symbol not in namelist
error when running netstat -rs.
Submitted by: Sebastian Huber <sebastian.huber@embedded-brains.de>
MFC after: 1 week
Sponsored by: Multiplay
invoked as llvm-ranlib, it can create an archive symbol table for
archives of objects compiled for LTO by an LLVM compiler.
Submitted by: Dan McGregor <danismostlikely@gmail.com>
MFC after: 3 days
Expand inet6name() line buffer to NI_MAXHOST and use strlcpy/snprintf
in various places.
Reported by: Anton Yuzhaninov <citrin citrin ru>
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D8916
It's been dead ever since it was imported from TI-RPC in 1995. The dead
code is still present in Illumos today, but was removed from NetBSD in 2006.
Reported by: Coverity
CID: 270097
Obtained from: NetBSD
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
It's always been dead, ever since first import in 1994. It's still dead in
OpenBSD's version, too.
Reported by: Coverity
CID: 270586
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
party software, this provides more standarized import workflow and
makes future upgrades easier.
The following files are new with this commit:
zconf.h.in
zlib.map
zlib.pc.in
They are not connected to build, but were kept in tree for reference
for future maintenance.
All our local trivial changes were applied to contrib/zlib, and the
contrib/zlib vendor source code is intended to 100% match lib/libz
before this commit.
MFC after: 2 weeks
As a followup to r310638, update libsysdecode (and kdump) to decode the
'mode' argument to getfsstat(). sysdecode_getfsstat_flags() has been
renamed to sysdecode_getfsstat_mode() and now treats the argument as an
enumerated value rather than a mask of flags.
Without this change, indent(1) would only look to load options from ~/.indent.pro if it's there and -npro wasn't used on the command line. This option lets the user set their own path to the file.
Approved by: pfg (mentor)
Differential Revision: https://reviews.freebsd.org/D9010
Move llvm-objdump from CLANG_EXTRAS to installed by default
We currently install three tools from binutils 2.17.50: as, ld, and
objdump. Work is underway to migrate to a permissively-licensed
tool-chain, with one goal being the retirement of binutils 2.17.50.
LLVM's llvm-objdump is intended to be compatible with GNU objdump
although it is currently missing some options and may have formatting
differences. Enable it by default for testing and further investigation.
It may later be changed to install as /usr/bin/objdump, it becomes a
fully viable replacement.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D8879
We currently install three tools from binutils 2.17.50: as, ld, and
objdump. Work is underway to migrate to a permissively-licensed
tool-chain, with one goal being the retirement of binutils 2.17.50.
LLVM's llvm-objdump is intended to be compatible with GNU objdump
although it is currently missing some options and may have formatting
differences. Enable it by default for testing and further investigation.
It may later be changed to install as /usr/bin/objdump, it becomes a
fully viable replacement.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D8879
zero-length array are dynamically sized at run-time based on the use
of hints, compilers can't be expected to figure out these offsets on
their own. [1]
- Fix incorrect comparison in cmp_nans(). [2]
PR: 204571 [1], 202301 [2]
Submitted by: David Binderman [2]
MFC after: 3 days
