Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
Small summary
-------------
o Almost all IPsec releated code was moved into sys/netipsec.
o New kernel modules added: ipsec.ko and tcpmd5.ko. New kernel
option IPSEC_SUPPORT added. It enables support for loading
and unloading of ipsec.ko and tcpmd5.ko kernel modules.
o IPSEC_NAT_T option was removed. Now NAT-T support is enabled by
default. The UDP_ENCAP_ESPINUDP_NON_IKE encapsulation type
support was removed. Added TCP/UDP checksum handling for
inbound packets that were decapsulated by transport mode SAs.
setkey(8) modified to show run-time NAT-T configuration of SA.
o New network pseudo interface if_ipsec(4) added. For now it is
build as part of ipsec.ko module (or with IPSEC kernel).
It implements IPsec virtual tunnels to create route-based VPNs.
o The network stack now invokes IPsec functions using special
methods. The only one header file <netipsec/ipsec_support.h>
should be included to declare all the needed things to work
with IPsec.
o All IPsec protocols handlers (ESP/AH/IPCOMP protosw) were removed.
Now these protocols are handled directly via IPsec methods.
o TCP_SIGNATURE support was reworked to be more close to RFC.
o PF_KEY SADB was reworked:
- now all security associations stored in the single SPI namespace,
and all SAs MUST have unique SPI.
- several hash tables added to speed up lookups in SADB.
- SADB now uses rmlock to protect access, and concurrent threads
can do SA lookups in the same time.
- many PF_KEY message handlers were reworked to reflect changes
in SADB.
- SADB_UPDATE message was extended to support new PF_KEY headers:
SADB_X_EXT_NEW_ADDRESS_SRC and SADB_X_EXT_NEW_ADDRESS_DST. They
can be used by IKE daemon to change SA addresses.
o ipsecrequest and secpolicy structures were cardinally changed to
avoid locking protection for ipsecrequest. Now we support
only limited number (4) of bundled SAs, but they are supported
for both INET and INET6.
o INPCB security policy cache was introduced. Each PCB now caches
used security policies to avoid SP lookup for each packet.
o For inbound security policies added the mode, when the kernel does
check for full history of applied IPsec transforms.
o References counting rules for security policies and security
associations were changed. The proper SA locking added into xform
code.
o xform code was also changed. Now it is possible to unregister xforms.
tdb_xxx structures were changed and renamed to reflect changes in
SADB/SPDB, and changed rules for locking and refcounting.
Reviewed by: gnn, wblock
Obtained from: Yandex LLC
Relnotes: yes
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D9352
Load kvm symbols earlier to prevent rstat: symbol not in namelist
error when running netstat -rs.
Submitted by: Sebastian Huber <sebastian.huber@embedded-brains.de>
MFC after: 1 week
Sponsored by: Multiplay
Expand inet6name() line buffer to NI_MAXHOST and use strlcpy/snprintf
in various places.
Reported by: Anton Yuzhaninov <citrin citrin ru>
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D8916
sendfile_swapin() loop works this way:
- Find first invalid page in the request.
- Do vm_pager_has_page() and get count of pages, that can be taken in
single I/O.
- Trim valid pages from the end of the request.
- Cycle through the request and substitute to bogus_page all valid
pages that are in the middle of the request.
- After I/O launched (pager copies array of pages into buf(9), it
is important to restore proper page pointers with help vm_page_lookup().
Count bogus pages used and report them in sendfile stats.
This fixes a bug with link local addresses. This will require and
upcoming change in the kernel to bring SCTP to the same behaviour
as UDP and TCP.
MFC after: 3 days
with the corresponding entry in the table header.
r295136 increased the value width from 14 to 32 without the corresponding
change to the table header. This commit adds the change to the table
header width.
MFC after: 3 days
In the case the width is less than 0, we are returning an uninitialized
value. For practical purposes the return value is ignored but initialize
it to avoid trouble.
CID: 1341619
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.
Sponsored by: EMC / Isilon Storage Division
up to now.
The new sendfile is the code that Netflix uses to send their multiple tens
of gigabits of data per second. The new implementation features asynchronous
I/O, when I/O operations are launched, but not awaited to be complete. An
explanation of why such behavior is beneficial compared to old one is
going to be too long for a commit message, so we will skip it here.
Additional features of new syscall are extra flags, which provide an
application more control over data sent. The SF_NOCACHE flag tells
kernel that data shouldn't be cached after it was sent. The SF_READAHEAD()
macro allows to specify readahead size in pages.
The new syscalls is a drop in replacement. No modifications are required
to applications. One can take nginx binary for stable/10 and run it
successfully on head. Although SF_NODISKIO lost its original sense, as now
sendfile doesn't block, and now means something completely different (tm),
using the new sendfile the old way is absolutely safe.
Celebrates: Netflix global launch!
Sponsored by: Nginx, Inc.
Sponsored by: Netflix
Relnotes: yes
variable set rather than reading from kernel memory.
This also makes the -z (zero) flag work correctly
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC (Netgate)
Differential Revision: https://reviews.freebsd.org/D4591
Unfortunately filemon/meta mode tracks all indirect dependencies here
since ld(1) is reading libelf when linking in libkvm. Churn would be
reduced if this was able to be limited to direct dependencies.
Sponsored by: EMC / Isilon Storage Division
This is so that 'make depend' is not a required build step in these
files.
DPSRCS is overall unneeded. DPSRCS already contains SRCS, so anything
which can safely be in SRCS should be. DPSRCS is mostly just a way to
generate files that should not be linked into the final PROG/LIB. For
headers and grammars it is safe for them to be in SRCS since they will
be excluded during linking and installation.
The only remaining uses of DPSRCS are for generating .c or .o files that
must be built before 'make depend' can run 'mkdep' on the SRCS c files
list. A semi-proper example is in tests/sys/kern/acct/Makefile where a
checked-in .c file has an #include on a generated .c file. The
generated .c file should not be linked into the final PROG though since
it is #include'd. The more proper way here is just to build/link it in
though without DPSRCS. Another example is in sys/modules/linux/Makefile
where a shell script runs to parse a DPSRCS .o file that should not be
linked into the module. Beyond those, the need for DPSRCS is largely
unneeded, redundant, and forces 'make depend' to be ran. Generally,
these Makefiles should avoid the need for DPSRCS and define proper
dependencies for their files as well.
An example of an improper usage and why this matters is in usr.bin/netstat.
nl_defs.h was only in DPSRCS and so was not generated during 'make all',
but only during 'make depend'. The files including it lacked proper
depenencies on it, which forced running 'make depend' to workaround that
bug. The 'make depend' target should mostly be used for incremental build
help, not to produce a working build. This specific example was broken in
the meta build until r287905 since it does not run 'make depend'.
The gnu/lib/libreadline/readline case is fine since bsd.lib.mk has 'OBJS:
SRCS:M*.h' when there is no .depend file.
Sponsored by: EMC / Isilon Storage Division
MFC after: 1 week
routepr() (-r flag). It is too narrow to show an IPv6 prefix
in most cases.
- Accept "local" as a synonym of "unix" in protocol family name.
- Show a prefix length in CIDR notation when name resolution failed in
netname().
- Make routename() and netname() AF-independent and remove
unnecessary typecasting from struct sockaddr.
- Use getnameinfo(3) to format L2 addr in intpr().
- Fix a bug which showed "Address" when -A flag is specfied in pr_rthdr().
- Replace cryptic GETSA() macro with SA_SIZE().
- Fix declarations shadowing local variables with the same names.
- Add more static, remove unused header files and variables.
MFC after: 1 week
In the kernel, structs such as tcpstat are manipulated as an array of
counter_u64_t (uint64_t *), but made visible to userland as an array of
uint64_t. kread_counters() was previously copying the counter array into
user space and sequentially overwriting each counter with its value. This
mostly affects IPsec counters, as other counters are exported via sysctl.
PR: 201700
Tested by: Jason Unovitch
MFC after: 1 week
Update setkey and libipsec to understand aes-gcm-16 as an
encryption method.
A partial commit of the work in review D2936.
Submitted by: eri
Reviewed by: jmg
MFC after: 2 weeks
Sponsored by: Rubicon Communications (Netgate)
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
o Restore historical behaviour of appending '*' if interface is down,
and we have enough space to print it (usually we don't). [1]
o Do not truncate interface names when printing in encoded format.
o Report interface flags into encoded format.
PR: 199873 [1]
Sponsored by: Nginx, Inc.
in 'netstat -r'.
The netstat/route.c was the last abuser of struct ifnet and struct
rtentry in the tree. With this change if_var.h can become kernel
only include, _WANT_RTENTRY can go away and projects/ifnet and
projects/routing can go forward.
Differential Revision: https://reviews.freebsd.org/D2242
Reviewed by: melifaro, gnn
Sponsored by: Nginx, Inc.
Sponsored by: Netflix
contain kernel pointers, and instead has interface index.
Bump __FreeBSD_version for that change.
o Now, netstat/mroute6.c no longer needs to kvm_read(3) struct ifnet, and
no longer needs to include if_var.h
Note that this change is far from being a complete move of IPv6 multicast
routing to a proper API. Other structures are still dumped into their
sysctls as is, requiring userland application to #define _KERNEL when
including ip6_mroute.h and then call kvm_read(3) to gather all bits and
pieces. But fixing this is out of scope of the opaque ifnet project.
Sponsored by: Nginx, Inc.
Sponsored by: Netflix
Obtained from: Phil Shafer <phil@juniper.net>
Ported to -current by: alfred@ (mostly), Kim Shrier
Formatting: marcel@
Sponsored by: Juniper Networks, Inc.
o Introduce a notion of "not ready" mbufs in socket buffers. These
mbufs are now being populated by some I/O in background and are
referenced outside. This forces following implications:
- An mbuf which is "not ready" can't be taken out of the buffer.
- An mbuf that is behind a "not ready" in the queue neither.
- If sockbet buffer is flushed, then "not ready" mbufs shouln't be
freed.
o In struct sockbuf the sb_cc field is split into sb_ccc and sb_acc.
The sb_ccc stands for ""claimed character count", or "committed
character count". And the sb_acc is "available character count".
Consumers of socket buffer API shouldn't already access them directly,
but use sbused() and sbavail() respectively.
o Not ready mbufs are marked with M_NOTREADY, and ready but blocked ones
with M_BLOCKED.
o New field sb_fnrdy points to the first not ready mbuf, to avoid linear
search.
o New function sbready() is provided to activate certain amount of mbufs
in a socket buffer.
A special note on SCTP:
SCTP has its own sockbufs. Unfortunately, FreeBSD stack doesn't yet
allow protocol specific sockbufs. Thus, SCTP does some hacks to make
itself compatible with FreeBSD: it manages sockbufs on its own, but keeps
sb_cc updated to inform the stack of amount of data in them. The new
notion of "not ready" data isn't supported by SCTP. Instead, only a
mechanical substitute is done: s/sb_cc/sb_ccc/.
A proper solution would be to take away struct sockbuf from struct
socket and allow protocols to implement their own socket buffers, like
SCTP already does. This was discussed with rrs@.
Sponsored by: Netflix
Sponsored by: Nginx, Inc.
It affects the IPv6 source address selection algorithm (RFC 6724)
and allows override the last rule ("longest matching prefix") for
choosing among equivalent addresses. The address with `prefer_source'
will be preferred source address.
Obtained from: Yandex LLC
MFC after: 1 month
Sponsored by: Yandex LLC
- Reformat the entire man page
- Create a proper synopsis section
- Use itemized-lists to describe each flag, rather than paragraphs
- Cross-reference common flags to a 'general flags' sub-section with short
inline description of the flag
- Label 'general flags' sub-section
- Apply additional fixes suggested by wblock, brueffer, and bdrewery
- Update .Dd that got undone previously
- Change the order of the .Op Fl to be alphabetical
- Add the -i | -I interface flags to the description of 'interface
display mode'
- Fix missing parameters in man page
- Fix missing parameters in usage()
- Sync man page and usage()
MFC Note: stable/9 and stable/10 do not have -R, will need to be removed
when merged
CR: D58
Reviewed by: brueffer, bcr
Approved by: wblock (mentor)
MFC after: 7 days
Sponsored by: ScaleEngine Inc.
- Increase WID_IF_DEFAULT() from 6 to 8 (the default for AF_INET6) because
we have interfaces with longer names than 6 chars like epairN{a,b}.
- Style fixes.
This is intended to help in diagnostics and debugging of NIC and stack
flowid support.
Eventually this will grow another column (RSS CPU ID) but
that currently isn't cached in the inpcb.
There's also no clean flowtype -> flowtype identifier string. This is
the mbuf M_HASHTYPE_* values for RSS.
Here's some example output:
adrian@adrian-hackbox:~/work/freebsd/head/src % netstat -Rn | more
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address flowid ftype
tcp4 0 0 10.11.1.65.22 10.11.1.64.12409 29041942 2
udp4 0 0 127.0.0.1.123 *.* 00000000 0
udp6 0 0 fe80::1%lo0.123 *.* 00000000 0
udp6 0 0 ::1.123 *.* 00000000 0
udp4 0 0 10.11.1.65.123 *.* 00000000 0
Tested:
* amd64 system w/ igb NIC; local driver changes to expose RSS flowid in if_igb.
same events that tcpstat's tcps_rcvmemdrop counter counts.
- Rename tcps_rcvmemdrop to tcps_rcvreassfull and improve its
description in netstat(1) output.
Sponsored by: Netflix
Sponsored by: Nginx, Inc.
netstat has two options for printing multicast tables:
sysctl (the default one for live systems) and kvm-based one (for cores).
It looks like kvm-based one hasn't been working since it's been introduced
in r190012 due to absence of mfctablesize kernel symbol.
Check for all ipv4-multicast symbols being correctly resolved was introduced
in r259638 regardless of 'live' value leading to "No IPv4 MROUTING" error
message.
Reported by: Olivier Cochard-Labbé
MFC after: 1 week
AppleTalk was a network transport protocol for Apple Macintosh devices
in 80s and then 90s. Starting with Mac OS X in 2000 the AppleTalk was
a legacy protocol and primary networking protocol is TCP/IP. The last
Mac OS X release to support AppleTalk happened in 2009. The same year
routing equipment vendors (namely Cisco) end their support.
Thus, AppleTalk won't be supported in FreeBSD 11.0-RELEASE.
