Commit Graph

6924 Commits

Author SHA1 Message Date
Alexander V. Chernikov
a666325282 Introduce nexthop objects and new routing KPI.
This is the foundational change for the routing subsytem rearchitecture.
 More details and goals are available in https://reviews.freebsd.org/D24141 .

This patch introduces concept of nexthop objects and new nexthop-based
 routing KPI.

Nexthops are objects, containing all necessary information for performing
 the packet output decision. Output interface, mtu, flags, gw address goes
 there. For most of the cases, these objects will serve the same role as
 the struct rtentry is currently serving.
Typically there will be low tens of such objects for the router even with
 multiple BGP full-views, as these objects will be shared between routing
 entries. This allows to store more information in the nexthop.

New KPI:

struct nhop_object *fib4_lookup(uint32_t fibnum, struct in_addr dst,
  uint32_t scopeid, uint32_t flags, uint32_t flowid);
struct nhop_object *fib6_lookup(uint32_t fibnum, const struct in6_addr *dst6,
  uint32_t scopeid, uint32_t flags, uint32_t flowid);

These 2 function are intended to replace all all flavours of
 <in_|in6_>rtalloc[1]<_ign><_fib>, mpath functions  and the previous
 fib[46]-generation functions.

Upon successful lookup, they return nexthop object which is guaranteed to
 exist within current NET_EPOCH. If longer lifetime is desired, one can
 specify NHR_REF as a flag and get a referenced version of the nexthop.
 Reference semantic closely resembles rtentry one, allowing sed-style conversion.

Additionally, another 2 functions are introduced to support uRPF functionality
 inside variety of our firewalls. Their primary goal is to hide the multipath
 implementation details inside the routing subsystem, greatly simplifying
 firewalls implementation:

int fib4_lookup_urpf(uint32_t fibnum, struct in_addr dst, uint32_t scopeid,
  uint32_t flags, const struct ifnet *src_if);
int fib6_lookup_urpf(uint32_t fibnum, const struct in6_addr *dst6, uint32_t scopeid,
  uint32_t flags, const struct ifnet *src_if);

All functions have a separate scopeid argument, paving way to eliminating IPv6 scope
 embedding and allowing to support IPv4 link-locals in the future.

Structure changes:
 * rtentry gets new 'rt_nhop' pointer, slightly growing the overall size.
 * rib_head gets new 'rnh_preadd' callback pointer, slightly growing overall sz.

Old KPI:
During the transition state old and new KPI will coexists. As there are another 4-5
 decent-sized conversion patches, it will probably take a couple of weeks.
To support both KPIs, fields not required by the new KPI (most of rtentry) has to be
 kept, resulting in the temporary size increase.
Once conversion is finished, rtentry will notably shrink.

More details:
* architectural overview: https://reviews.freebsd.org/D24141
* list of the next changes: https://reviews.freebsd.org/D24232

Reviewed by:	ae,glebius(initial version)
Differential Revision:	https://reviews.freebsd.org/D24232
2020-04-12 14:30:00 +00:00
Xin LI
f5b7695d2d Always install backward compatibility timezones, as they are installed
on all major Linux distributions as well as NetBSD and OpenBSD.

Remove the undocumented ZONEINFO_OLD_TIMEZONES_SUPPORT and the deprecated
OLDTIMEZONES knobs as they are now the default.

Reviewed by:		ngie, rgrimes
MFC after:		2 weeks
Differential Revision:	https://reviews.freebsd.org/D24306
2020-04-09 05:11:18 +00:00
David Bright
0e3e53d746 Add a basic test for nvmecontrol
I recently made some bug fixes in nvmecontrol. It occurred to me that
since nvmecontrol lacks any kyua tests, I should convert the informal
testing I did into a more formal automated test. The test in this
change should be considered just a starting point; it is neither
complete nor thorough. While converting the test to ATF/kyua, I
discovered a small bug in nvmecontrol; the nvmecontrol devlist command
would always exit with an unsuccessful status. So I included the fix
for that, too, so that the test won't fail.

Reviewed by:	imp@
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D24269
2020-04-07 20:26:42 +00:00
Maxim Sobolev
a8b8edb25e Normalize deployment tools usage and definitions by putting into one place
instead of sprinkling them out over many disjoint files. This is a follow-up
to achieve the same goal in an incomplete rev.348521.

Approved by:	imp
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20520
2020-04-07 02:46:22 +00:00
Brooks Davis
cfe7aad9c3 Create and use a tests group for the tests user.
No user (except nobody) should be a member of the nobody group.

Reported by:	rgrimes
Reviewed by:	rgrimes
MFC after:	3 days
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D24199
2020-03-27 16:05:37 +00:00
Brooks Davis
daa006f84b Install expected kyua docs.
The "kyua about" command assumes these files exist causing tests
supplied devel/kyua to fail.

Fix a bug defining the default KYUA_DOCDIR so the installed files can be
found.

Reported by:	jenkins tests
Reviewed by:	lwhsu
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D24187
2020-03-26 17:34:17 +00:00
Emmanuel Vadot
c016c89196 pkgbase: Move device.hints from the runtime to the bootloader package
Also mark it as config file so if a user changes this file pkg will attempt
to merge the new file upon an update.
device.hints is neither related to runtime or loader but it make more sense
to have it in loader in case some user delete /boot/ and wants to recreate it,
now only two packages are required FreeBSD-bootloader and the kernel package.
While here change where we override the package for files installed in /boot,
this allow us to keep other tags (such as config).

Reported by:	pizzamig
Reviewed by:	bapt pizzamig emaste
Differential Revision:	https://reviews.freebsd.org/D24159
2020-03-24 01:07:01 +00:00
Brooks Davis
b0d29bc47d Import the kyua test framework.
Having kyua in the base system will simplify automated testing in CI and
eliminates bootstrapping issues on new platforms.

The build of kyua is controlled by WITH(OUT)_TESTS_SUPPORT.

Reviewed by:	emaste
Obtained from:	CheriBSD
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D24103
2020-03-23 19:01:23 +00:00
Brooks Davis
d41872f999 Add the tests user, an unprivileged user from the default kyua config.
This is a preparatory commit for D24103.

Reviewed by:	emaste
Obtained from:	CheriBSD
MFC after:	3 days
Sponsored by:	DARPA
2020-03-23 17:11:27 +00:00
Ed Maste
4e3984bb5d BSD.sendmail.dist: simplify mqueue group name setting
Submitted by:	gshapiro
2020-03-22 16:55:20 +00:00
Ed Maste
7ab51667a7 pkgbase: create sendmail directories only from BSD.sendmail.dist
Else when WITHOUT_SENDMAIL is set we still create a sendmail package
that contains (only) two directories.

Reviewed by:	manu
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24149
2020-03-22 15:37:38 +00:00
Dimitry Andric
e43d33d286 Merge ^/head r358466 through r358677. 2020-03-05 17:55:36 +00:00
Warner Losh
daba5ace03 Finish removal of bktr
Remove the old ioctl .h files
Remove copying/linking ioctl .h files in instasllworld
Remove bktr from lint
Add now-removed files with ObsoleteFiles
2020-03-01 20:37:42 +00:00
Dimitry Andric
91019ea7d4 Merge ^/head r358400 through r358465. 2020-02-29 15:08:14 +00:00
Ed Maste
57f804675e remove GCC 4.2.1 build infrastructure
As described in Warner's email message[1] to the FreeBSD-arch mailing
list we have reached GCC 4.2.1's retirement date.  At this time all
supported architectures either use in-tree Clang, or rely on external
toolchain (i.e., a contemporary GCC version from ports).

GCC 4.2.1 was released July 18, 2007 and was imported into FreeBSD later
that year, in r171825.  GCC has served us well, but version 4.2.1 is
obsolete and not used by default on any architecture in FreeBSD.  It
does not support modern C and does not support arm64 or RISC-V.

Thanks to everyone responsible for maintaining, updating, and testing
GCC in the FreeBSD base system over the years.

So long, and thanks for all the fish.

[1] https://lists.freebsd.org/pipermail/freebsd-arch/2020-January/019823.html

PR:		228919
Reviewed by:	brooks, imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23124
2020-02-29 03:25:51 +00:00
Dimitry Andric
ef1c128c05 Merge ^/head r357921 through r357930. 2020-02-14 19:33:48 +00:00
Dimitry Andric
c42c3abb9e Remove /usr/include/ssp from BSD.include.dist after r356356
This avoids having to delete it every time with "make delete-old".

PR:		242950
MFC after:	2 weeks
X-MFC-With:	r356356
2020-02-14 19:31:24 +00:00
Dimitry Andric
bb1d0df511 Merge ^/head r357179 through r357269. 2020-01-29 21:09:36 +00:00
Kyle Evans
fbd46fe94a pkgbase: fix caroot packaging and add post-install script
The original intention for caroot was to be packaged separately, perhaps so
that users can have a more/less conservative upgrade policy for this
separated from the rest of base.

secure/caroot/Makefile doesn't have anything interesting to package, but its
subdirectories might. Move the PACKAGE= to Makefile.inc so both blacklisted
and trusted get packaged consistently into the correct one rather than the
default -utilities. Also tag the directories for package=caroot, as they
could also be empty; blacklisted is empty by default, but trusted is not.

Add a post-install script to do certctl rehash, along with a note should we
eventually come up with a way to detect that files have been added or
removed that requires a rehash.

-caroot gets a dependency on -utilities, as that's where we provide certctl
at the moment. We can perhaps reconsider this and put certctl into this
package in the future, but there are some bits within -utilities that
unconditionally invoke certctl so let's hold off for now.

Reviewed by:	manu (earlier version, before -utilities dep added)
Differential Revision:	https://reviews.freebsd.org/D23352
2020-01-29 18:47:08 +00:00
Dimitry Andric
0b37c15904 * Bump version numbers to 10.0.0
* Update UPDATING
* Update (Optional)ObsoleteFiles.inc
* Update VCS(Revision|Version) files
* Update generated config headers
* Update clang internal headers Makefile
2020-01-25 16:23:49 +00:00
Ed Maste
d3c87514db Tag NLS aliases with pkgbase package
POSIX and en_US.US_ASCII are aliases (symlinks) to the C locale.  They were
not previously tagged with a pkgbase pacakge.  Add the tag so that they are
handled correctly on pkgbase-installed/updated systems.

[This is r356990 reapplied with a corrected commit message.]

Discussed with:	manu
2020-01-24 14:41:51 +00:00
Ed Maste
3407334ae2 revert r356990 to reapply with correct commit message
Requested by:	rgrimes
2020-01-24 14:40:37 +00:00
Ed Maste
6ed1dc004a Tag NLS aliases with package=runtime
POSIX and en_US.US_ASCII are aliases (symlinks) to the C locale.  They were
not previously tagged with a pkgbase pacakge.  Add the tag so that they are
handled correctly on pkgbase-installed/updated systems.

Discussed with:	manu
2020-01-22 18:40:19 +00:00
Kyle Evans
156ebad647 Tag os-release symlink with package=runtime
This ensures it gets into pkgbase-(installed/updated) systems.
2020-01-22 14:07:27 +00:00
Jung-uk Kim
a9e3baa562 Install man5 and man7 for OpenSSL.
Note config.5 and crypto.7 are not installed because we have conflicts.

Requested by:	phk
MFC after:	1 month
2020-01-22 01:15:57 +00:00
Dimitry Andric
c14a5a8800 Merge llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and openmp
9.0.1 final release c1a0a213378a458fbea1a5c77b315c7dce08fd05.

Release notes for llvm, clang, lld and libc++ 9.0.1 will become
available here:

https://releases.llvm.org/9.0.1/docs/ReleaseNotes.html
https://releases.llvm.org/9.0.1/tools/clang/docs/ReleaseNotes.html
https://releases.llvm.org/9.0.1/tools/lld/docs/ReleaseNotes.html
https://releases.llvm.org/9.0.1/projects/libcxx/docs/ReleaseNotes.html

PR:		240629
MFC after:	1 month
2019-12-22 11:50:44 +00:00
Alan Somers
67f72211dd gmultipath: add ATF tests
Add ATF tests for most gmultipath operations. Add some dtrace probes too,
primarily for configuration changes that happen in response to provider
errors.

PR:		178473
MFC after:	2 weeks
Sponsored by:	Axcient
Differential Revision:	https://reviews.freebsd.org/D22235
2019-12-06 00:12:14 +00:00
Warner Losh
6467506baf Create /etc/os-release file.
Each boot, regenerate /var/run/os-release based on the currently running
system. Create a /etc/os-release symlink pointing to this file (so that this
doesn't create a new reason /etc can not be mounted read-only).

This is compatible with what other systems do and is what the sysutil/os-release
port attempted to do, but in an incomplete way. Linux, Solaris and DragonFly all
implement this natively as well. The complete standard can be found at
https://www.freedesktop.org/software/systemd/man/os-release.html

Moving this to the base solves both the non-standard location problem with the
port, as well as the lack of update of this file on system update.

Bump __FreeBSD_version to 1300060

PR: 238953
Differential Revision:  https://reviews.freebsd.org/D22271
2019-11-20 23:45:31 +00:00
Warner Losh
599d032a6d Standardize EFI's ESP mount point.
Mount the UEFI ESP on /boot/efi. No current system uses this by default, but
there are many ad-hoc schemes that do this in /efi or /esp or /uefi and adding a
new directory at the top-level would have a much higher likelihood of
collision. Document this in /etc/mtree/BSD.root.mtree and create EFIDIR and
related variables in bsd.own.mk.

Differential Revision: https://reviews.freebsd.org/D21344
2019-11-20 21:06:29 +00:00
Conrad Meyer
470182bb6e Link in NetBSD's unifdef(1) tests
Skip one, is it currently fails.
2019-11-18 04:03:11 +00:00
Conrad Meyer
43e9023f2d libexecinfo: Integrate NetBSD test into FreeBSD 2019-10-27 05:32:08 +00:00
Bjoern A. Zeeb
f74e6e494f frag6: import a set of test cases
In order to ensure that changing the frag6 code does not change behaviour
or break code a set of test cases were implemented.

Like some other test cases these use Scapy to generate packets and possibly
wait for expected answers.  In most cases we do check the global and
per interface (netstat) statistics output using the libxo output and grep
to validate fields and numbers.  This is a bit hackish but we currently have
no better way to match a selected number of stats only (we have to ignore
some of the ND6 variables; otherwise we could use the entire list).

Test cases include atomic fragments, single fragments, multi-fragments,
and try to cover most error cases in the code currently.
In addition vnet teardown is tested to not panic.

A separate set (not in-tree currently) of probes were used in order to
make sure that the test cases actually test what they should.

The "sniffer" code was copied and adjusted from the netpfil version
as we sometimes will not get packets or have longer timeouts to deal with.

Sponsored by:	Netflix
2019-10-21 09:33:45 +00:00
Dimitry Andric
8b3bc70a2b Merge ^/head r352764 through r353315. 2019-10-08 18:17:02 +00:00
Kyle Evans
f27f39db77 [1/3] Initial infrastructure for SSL root bundle in base
This setup will add the trusted certificates from the Mozilla NSS bundle
to base.

This commit includes:
- CAROOT option to opt out of installation of certs
- mtree amendments for final destinations
- infrastructure to fetch/update certs, along with instructions

A follow-up commit will add a certctl(8) utility to give the user control
over trust specifics. Another follow-up commit will actually commit the
initial result of updatecerts.

This work was done primarily by allanjude@, with minor contributions by
myself.

No objection from:	secteam
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D16856
2019-10-02 01:05:29 +00:00
Dimitry Andric
61c1328eb0 Merge ^/head r352105 through r352307. 2019-09-13 21:15:01 +00:00
Michael Zhilin
cd38a86c63 [jail] removal by jid doesn't trigger pre/post stop scripts
This commit fixes bug: command "jail -r" didn't trigger pre/post stop
commands (and others) defined in config file if jid is specified insted of
name. Also it adds basic tests for usr.sbin/jail to avoid regression.

Reviewed by:	jamie, kevans, ray
MFC after:      5 days
Differential Revision: https://reviews.freebsd.org/D21328
2019-09-12 18:53:29 +00:00
Yuri Pankov
42249ef234 locale: handle day, abday, mon, abmon, am_pm keywords
All of these are defined as mandatory by POSIX.

While here, mark all non-standard ones as FreeBSD-only as
other systems (at least, GNU/Linux and illumos) do not handle
them, so we should not encourage their use.

PR:		237752
Reviewed by:	bapt
Differential Revision:	https://reviews.freebsd.org/D21490
2019-09-10 15:09:46 +00:00
Dimitry Andric
f993ed2fbd Merge ^/head r351732 through r352104. 2019-09-09 19:58:46 +00:00
Kyle Evans
b27524973c patch(1): add some basic tests
Summary:
- basic: test application of patches created by diff -u at the
  beginning/middle/end of file, which have differing amounts of context
  before and after chunks being added
- limited_ctx: stems from PR 74127 in which a rogue line was getting added
  when the patch should have been rejected. Similar behavior was
  reproducible with larger contexts near the beginning/end of a file. See
  r326084 for details
- file_creation: patch sourced from /dev/null should create the file
- file_nodupe: said patch sourced from /dev/null shouldn't dupe the contents
  when re-applied (personal vendetta, WIP, see comment)
- file_removal: this follows from nodupe; the reverse of a patch sourced
  from /dev/null is most naturally deleting the file, as is expected based
  on GNU patch behavior (WIP)
2019-09-05 03:16:14 +00:00
Kris Moore
59e50df3cd - Retire pc-sysinstall(8)
https://reviews.freebsd.org/D21094

Submitted by: kmoore@FreeBSD.org
Approved by: imp@FreeBSD.org
2019-09-03 19:42:04 +00:00
Dimitry Andric
c5c3ba6b43 Merge ^/head r351317 through r351731. 2019-09-03 05:58:43 +00:00
Dimitry Andric
1db4d87c40 Fix mtree syntax. 2019-09-01 19:13:20 +00:00
Dimitry Andric
2f3724e658 Update Makefile and mtree for clang internal headers, and update
OptionalObsoleteFiles.inc for added and removed files.
2019-08-31 18:04:33 +00:00
Hiroki Sato
3eeebb94f1 Fix build (r351481). 2019-08-25 19:27:14 +00:00
Mateusz Piotrowski
babc4e8145 mixer(8): Report an error if the passed value is an empty string
This patch fixes a bug that made the mixer command enter
an infinite loop when instructed to set the value of a device
to an empty string (e.g., `mixer vol ""`).

Additionally, some tests for mixer(8) are being added.

PR:		240039
Reviewed by:	hselasky, mav
Approved by:	src (hselasky, mav)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21409
2019-08-25 17:55:31 +00:00
Dimitry Andric
72e1e27118 Change clang lib dir to 9.0.0. 2019-08-23 22:12:20 +00:00
Alan Somers
8e0f76c8f4 ping6: add a basic functional test
Submitted by:	Ján Sučan <sucanjan@gmail.com>
MFC after:	2 weeks
Sponsored by:	Google, inc. (Google Summer of Code 2019)
Differential Revision:	https://reviews.freebsd.org/D21292
2019-08-22 15:08:04 +00:00
Alan Somers
c43633c6a1 ping: Add tests of the Internet checksum function
Submitted by:	Ján Sučan <sucanjan@gmail.com>
MFC after:	2 weeks
Sponsored by:	Google LLC (Google Summer of Code 2019)
Differential Revision:	https://reviews.freebsd.org/D21340
2019-08-20 21:59:48 +00:00
Alan Somers
0b4275accb fusefs: merge from projects/fuse2
This commit imports the new fusefs driver. It raises the protocol level
from 7.8 to 7.23, fixes many bugs, adds a test suite for the driver, and
adds many new features. New features include:

* Optional kernel-side permissions checks (-o default_permissions)
* Implement VOP_MKNOD, VOP_BMAP, and VOP_ADVLOCK
* Allow interrupting FUSE operations
* Support named pipes and unix-domain sockets in fusefs file systems
* Forward UTIME_NOW during utimensat(2) to the daemon
* kqueue support for /dev/fuse
* Allow updating mounts with "mount -u"
* Allow exporting fusefs file systems over NFS
* Server-initiated invalidation of the name cache or data cache
* Respect RLIMIT_FSIZE
* Try to support servers as old as protocol 7.4

Performance enhancements include:

* Implement FUSE's FOPEN_KEEP_CACHE and FUSE_ASYNC_READ flags
* Cache file attributes
* Cache lookup entries, both positive and negative
* Server-selectable cache modes: writethrough, writeback, or uncached
* Write clustering
* Readahead
* Use counter(9) for statistical reporting

PR:		199934 216391 233783 234581 235773 235774 235775
PR:		236226 236231 236236 236291 236329 236381 236405
PR:		236327 236466 236472 236473 236474 236530 236557
PR:		236560 236844 237052 237181 237588 238565
Reviewed by:	bcr (man pages)
Reviewed by:	cem, ngie, rpokala, glebius, kib, bde, emaste (post-commit
		review on project branch)
MFC after:	3 weeks
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Pull Request:	https://reviews.freebsd.org/D21110
2019-08-07 00:38:26 +00:00
Tom Jones
f97a8a3615 Add common firewall test suite
Add a common test suite for the firewalls included in the base system. The test
suite allows common test infrastructure to test pf, ipfw and ipf firewalls from
test files containing the setup for all three firewalls.

Add the pass block test for pf, ipfw and ipf. The pass block test checks the
allow/deny functionality of the firewalls tested.

Submitted by:   Ahsan Barkati
Sponsored by:   Google, Inc. (GSoC 2019)
Reviewed by:    kp
Approved by:    bz (co-mentor)
MFC after:      2 weeks
Differential Revision: https://reviews.freebsd.org/D21065
2019-08-05 11:47:34 +00:00