1. separating L2 tables (ARP, NDP) from the L3 routing tables
2. removing as much locking dependencies among these layers as
possible to allow for some parallelism in the search operations
3. simplify the logic in the routing code,
The most notable end result is the obsolescent of the route
cloning (RTF_CLONING) concept, which translated into code reduction
in both IPv4 ARP and IPv6 NDP related modules, and size reduction in
struct rtentry{}. The change in design obsoletes the semantics of
RTF_CLONING, RTF_WASCLONE and RTF_LLINFO routing flags. The userland
applications such as "arp" and "ndp" have been modified to reflect
those changes. The output from "netstat -r" shows only the routing
entries.
Quite a few developers have contributed to this project in the
past: Glebius Smirnoff, Luigi Rizzo, Alessandro Cerri, and
Andre Oppermann. And most recently:
- Kip Macy revised the locking code completely, thus completing
the last piece of the puzzle, Kip has also been conducting
active functional testing
- Sam Leffler has helped me improving/refactoring the code, and
provided valuable reviews
- Julian Elischer setup the perforce tree for me and has helped
me maintaining that branch before the svn conversion
when the CC is not set. Note NO_COUNTRY is set to 0xffff for now
(must be 16 bits as ieee80211_regdomain struct defines sku's and
cc's as uint16_t which may need fixing).
o do not require 1/2 and 1/4 rate channels be present in the
calibration list when doing a gsm regulatory change; the
existing 900MHz cards are not self-identifying so there is
no way (using the calibration channel list) to check
o store XML_Parser in the state block so we can report line numbers for errors
o complain about netband w/o mode
o complain about unknown modes
o complain about band w/o enclosing netband
o complain about duplicate freqband
o complain about unknown channel flags
o complain about band w/o freqband's
o complain about band w/o maxpower
o complain about country w/o ISO cc
o complain about country w/o regdomain reference
might do subsequent reads from other providers. This stopped geli (and
probably other classes using g_metadata_store as well) from being put on top
of gvinum raid5 volumes.
Note:
The reason it fails in the gvinum raid5 case is that gvinum will read back the
old parity stripe before calculating the new parity stripe to be written out
again. The write will then fail because the underlying disk to be read is
opened write only.
MFC after: 1 week
it was before -- the check is only made when getdisklabel()
returns valid info.
On passing, use MAXPARTITIONS to identify the max partition number,
instead of the hardwired 'h'
MFC after: 4 weeks
want to prepare disk images for emulators (though 'makefs' in port
can do something similar).
This relies on:
+ minor changes to pass the consistency checks even when working on a file;
+ an additional option, '-p partition' , to specify the disk partition to
initialize;
+ some changes on the I/O routines to deal with partition offsets.
The latter was a bit tricky to implement, see the details in newfs.h:
in newfs, I/O is done through libufs which assumes that the file
descriptor refers to the whole partition. Introducing support for
the offset in libufs would require a non-backward compatible change
in the library, to be dealt with a version bump or with symbol
versioning.
I felt both approaches to be overkill for this specific application,
especially because there might be other changes to libufs that might
become necessary in the near future.
So I used the following trick:
- read access is always done by calling bread() directly, so we just add
the offset in the (few) places that call bread();
- write access is done through bwrite() and sbwrite(), which in turn
calls bwrite(). To avoid rewriting sbwrite(), we supply our own version
of bwrite() here, which takes precedence over the version in libufs.
MFC after: 4 weeks
when preparing images for emulators or flash devices:
+ option '-C size' to create the underlying image file with given size.
Saves doing a 'dd' before, and especially it creates a sparse file
+ option '-@ offset' to build the FAT image at the specified offset
in the image file or device;
+ make the cluster size adaptive on the filesystem size.
Previously the default was 4k which is really unconvenient with
large media; now it goes from 512 bytes to 32k depending on
filesystem size (i still need to check whether it makes sense
to go further up, to 64k or above);
+ fix default geometry when not specified on the command line,
use 63 sectors/255 heads by default.
Also trim the size so it exactly a multiple of a track, to avoid
complaints in some filesystem code.
+ document all the above, plus some manual page clarifications.
MFC after: 4 weeks
mount_msdosfs: /dev/cf0s1: : Operation not supported by device
and thought I'd fix it to be:
mount_msdosfs: /dev/cf0s1: Operation not supported by device
Not sure why errmsg isn't getting filled in, or why this error is even
happening at all... (fsck_msdosfs is clean, and I can mount this same
CF elsewhere).
a plain file and a geometry is not explicitly supplied through
command line or disktab entry.
This way you can a FAT image on a file as simply as this:
newfs_msdos ./some/file
(right now you need a much longer command
newfs_msdos -h 32 -u 64 -S 512 -s $total_blocks -o 0 ./some/file
Will be merged after 7.1 and 6.4 are released.
See also the related PR which suggests a similar change.
PR: bin/121182
MFC after: 4 weeks
many bugs fixes, many more performance improvements.
Submitted by: Danny Braniss
M sbin/iscontrol/iscsi.conf.5
M sbin/iscontrol/iscontrol.8
M sbin/iscontrol/iscontrol.h
M sbin/iscontrol/config.c
M sbin/iscontrol/fsm.c
M sbin/iscontrol/login.c
M sbin/iscontrol/pdu.c
M sbin/iscontrol/misc.c
M sbin/iscontrol/auth_subr.c
M sbin/iscontrol/iscontrol.c
M sys/dev/iscsi/initiator/isc_cam.c
M sys/dev/iscsi/initiator/iscsi.h
M sys/dev/iscsi/initiator/isc_soc.c
M sys/dev/iscsi/initiator/iscsi_subr.c
M sys/dev/iscsi/initiator/iscsivar.h
M sys/dev/iscsi/initiator/isc_subr.c
M sys/dev/iscsi/initiator/iscsi.c
M sys/dev/iscsi/initiator/isc_sm.c
formatting a number in a human-friendly way.
Note that with this commit a megabyte changed from 1000000 to
1048576 and a 80G disk is now printed as being 75G in size.
This is deliberate. It's consistent with the core of geom(8).
However, the original choice for a megabyte being 1000000 was
on purpose and matches what disk vendors put on the box. The
consistency is considered more important.
Submitted by: delphij
and server. This replaces the RPC implementation of the NFS client and
server with the newer RPC implementation originally developed
(actually ported from the userland sunrpc code) to support the NFS
Lock Manager. I have tested this code extensively and I believe it is
stable and that performance is at least equal to the legacy RPC
implementation.
The NFS code currently contains support for both the new RPC
implementation and the older legacy implementation inherited from the
original NFS codebase. The default is to use the new implementation -
add the NFS_LEGACYRPC option to fall back to the old code. When I
merge this support back to RELENG_7, I will probably change this so
that users have to 'opt in' to get the new code.
To use RPCSEC_GSS on either client or server, you must build a kernel
which includes the KGSSAPI option and the crypto device. On the
userland side, you must build at least a new libc, mountd, mount_nfs
and gssd. You must install new versions of /etc/rc.d/gssd and
/etc/rc.d/nfsd and add 'gssd_enable=YES' to /etc/rc.conf.
As long as gssd is running, you should be able to mount an NFS
filesystem from a server that requires RPCSEC_GSS authentication. The
mount itself can happen without any kerberos credentials but all
access to the filesystem will be denied unless the accessing user has
a valid ticket file in the standard place (/tmp/krb5cc_<uid>). There
is currently no support for situations where the ticket file is in a
different place, such as when the user logged in via SSH and has
delegated credentials from that login. This restriction is also
present in Solaris and Linux. In theory, we could improve this in
future, possibly using Brooks Davis' implementation of variant
symlinks.
Supporting RPCSEC_GSS on a server is nearly as simple. You must create
service creds for the server in the form 'nfs/<fqdn>@<REALM>' and
install them in /etc/krb5.keytab. The standard heimdal utility ktutil
makes this fairly easy. After the service creds have been created, you
can add a '-sec=krb5' option to /etc/exports and restart both mountd
and nfsd.
The only other difference an administrator should notice is that nfsd
doesn't fork to create service threads any more. In normal operation,
there will be two nfsd processes, one in userland waiting for TCP
connections and one in the kernel handling requests. The latter
process will create as many kthreads as required - these should be
visible via 'top -H'. The code has some support for varying the number
of service threads according to load but initially at least, nfsd uses
a fixed number of threads according to the value supplied to its '-n'
option.
Sponsored by: Isilon Systems
MFC after: 1 month
background fsck on the same file system might then print negative
numbers for reclaimed directories/files/fragments.
Address the issue in a limited degree, by using old summary data for
cg when bgfsck is performed.
Submitted by: tegge
MFC after: 1 week
systems less than 1 TB, due to using 32-bits integers for file system block
numbers. This also causes incorrect error reporting for foreground fsck.
Convert it to use ufs2_daddr_t for block numbers.
PR: kern/127951
Submitted by: tegge
MFC after: 1 week
all others. Use this to disambiguate cmd line arguments that can
be either clone params or regular parameters so, in particular,
"bssid" again works as a regular parameter.
While here leverage the above to improve the logic for flushing
clone operations on the first !clone cmd line parameter.
Reviewed by: jhay
control over the result of buildworld and installworld; this especially
helps packaging systems such as nanobsd
Reviewed by: various (posted to arch)
MFC after: 1 month
size and the sector size.
- Fix a bug where bsdlabel would try to read a regular file using the geom_bsd
class.
Quick review by: phk
Approved by: pjd (mentor)
via nmount(), build up an iovec where each iovec member is an NFS mount
option, and pass the iovec down to the kernel via nmount(). These options
are then parsed in the kernel.
This should make it easier to add new NFS mount options in future.
Many, many thanks to Doug Rabson for taking my initial patches,
and cleaning them up. In addition, Doug added a fallback_mount()
function so that the newer mount_nfs program will work against older
kernels, to facilitate upgrading/downgrading scenarios.
Doug also re-wrote the mount_nfs.8 man page.
Reviewed by: dfr
duplication of code in fdisk and boot0cfg. Also make use of g_providername to
fix an issue with fdisk and boot0cfg not using the correct provider when
writing the MBR.
Reviewed by: phk
Approved by: pjd (mentor)
once it is lost, all data is gone.
Option '-B none' can by used to prevent backup. Option '-B path' can be
used to backup metadata to a different file than the default, which is
/var/backups/<prov>.eli.
The 'geli init' command also prints backup file location and gives short
procedure how to restore metadata.
The 'geli setkey' command now warns that even after passphrase change or keys
update there could be version of the master key encrypted with old
keys/passphrase in the backup file.
Add regression tests to verify that new functionality works as expected.
Update other regression tests so they don't create backup files.
Reviewed by: keramida, rink
Dedicated to: a friend who lost 400GB of his live by accidentally overwritting geli metadata
MFC after: 2 weeks
ping(8)'s -a was mapped to -e, but -E was already taken in ping6 (old
option) so rename -e to -r.
Now:
ping -a => ping6 -r
ping -A => ping6 -R
MFC after: 2 days
userspace to kernel via nmount(), pass in the strings
"update", "snapshot", "reload".
We want to move away from passing MNT_ flags from userspace -> kernel
via nmount(), and instead favor passing the string options.
IP_MULTICAST_IF with struct ip_mreqn (obtained from Linux) to tell the
stack which interface index to use for sending IPv4 datagrams.
Submitted by: bms
Tested by: phk
obsoleted by gpart(8). This avoids the following bugs in fdisk:
- initializing a disk without MBR bogusly emits the error:
fdisk: invalid fdisk partition table found
- initializing a disk with or without MBR bogusly emits either:
fdisk: Class not found
or
fdisk: Geom not found: "XXX"
- the default geometry for non-ATA and non-SCSI disks is either
invalid or sub-optimizal.
RELEASE_CRUNCH builds use NO_MAN anyway, so this change is primarily
to avoid that developers have to set NO_MAN manually when they build
the static variant.
In the MPSAFE TTY branch, I noticed PTY's to be leaked, because
dhclient's privileged process was run inside the session of, say, the
login shell. Make sure we call setsid() here.
Approved by: philip (mentor), brooks
return a pointer to a void. The send_thread() and disk_thread()
funtions; however, do not have a return value because they run for
the duration of the daemon's lifetime. This causes gcc to barf when
running with -O3. Make these functions return a null pointer to quiet it.
PR: bin/124342
Submitted by: Garrett Cooper <gcooper@FreeBSD.org> (minus his comments)
MFC after: 1 week
others. In the case where it displayed warnings it would still return
succesfully. Modify it so that it returns the number of sysctls that
it was not able to set.
Make use of this in rc.d to display only *unsuccessfull* attempts to
set sysctls.
The -l option changes the output to show the partition label, if applicable
and when present. The -r option changes the output to show the raw (i.e.
scheme-specific) partition types.
o gctl_delete_param() -- intended for parameters that are consumed
by geom(8) itself and which should not be passed to the kernel.
o gctl_has_param() -- intended to check if optional parameters are
present.
Both are needed by gpart(8) to process the optional parameter for
writing bootcode to a partition (as part of the bootcode verb).
However, the kernel is itself not involved in this matter and the
parameter needs to be removed from the request destined for the
kernel.
and define STATIC_GEOM_CLASSES when building the rescue binary. This way
geom can more easily be part of other crunched binaries, as it requires
only a Makefile change.
and be usable in scripts, etc.
This also changes the semantics in case when we lose one of n packets.
In that case, before we exited by SIG, now we exit with return(0).
Submitted by: Gert Doering (gert space.net)
MFC after: 10 days
NET_NEEDS_GIANT. netatm has been disconnected from the build for ten
months in HEAD/RELENG_7. Specifics:
- netatm include files
- netatm command line management tools
- libatm
- ATM parts in rescue and sysinstall
- sample configuration files and documents
- kernel support as a module or in NOTES
- netgraph wrapper nodes for netatm
- ctags data for netatm.
- netatm-specific device drivers.
MFC after: 3 weeks
Reviewed by: bz
Discussed with: bms, bz, harti
to a remote machine, the fact that the dump date is stored with
each header (inode) record makes rsync significantly less efficient
than necessary. This also applies to inode access times when they
are not important data to retain. When implementing an offsite
backup solution of this type, these dates in particular are not
important, especially if it prevents effective offsite backups.
PR: bin/91049
Submitted by: Forrest W Christian <fwc@mt.net>
on a 7.0 or later system that were created on a pre-5.0 system.
We must ensure that restore zeros out the previously undefined
birthtime and external attribute size fields when reading dump
tapes made by the UFS1 dump program.
The problem is that UFS2 dump carefully zeros out the unused
birthtime and external attribute size fields in the dump header
when dumping UFS1 filesystems, but the UFS1 dump didn't know about
those fields (they were spares) so just left whatever random junk
was in them. So, when restoring one of these pre-UFS2 dumps,
the new restore would eventually trip across a header that had
a non-zero external attribute size and try to extract it. That
consumed several tape blocks which left it totally out of sync
and very unhappy (i.e., the panic). The fix is in the gethead()
function which modernizes old headers by copying old fields to
their new location (and with this fix) zeroing out previously
undefined fields.
PR: bin/120881
Review by: David Malone & Scott Lambert
MFC after: 1 week
Must ensure that dump tapes from UFS1 filesystems properly copy
old fields of dump headers to new locations. Move check of dumpdate
to follow the code which ensures that the appropriate fields have
been copied.
PR: bin/118087
Help from: David Malone, Scott Lambert, Javier Martín Rueda
MFC after: 2 weeks
giving the output in a human-readable form. This behaviour is consistent
with most of system tools.
- Add -m and -g options to give output in megabytes and gigabytes
respectively.
This particular implementation is designed to be fully backwards compatible
and to be MFC-able to 7.x (and 6.x)
Currently the only protocol that can make use of the multiple tables is IPv4
Similar functionality exists in OpenBSD and Linux.
From my notes:
-----
One thing where FreeBSD has been falling behind, and which by chance I
have some time to work on is "policy based routing", which allows
different
packet streams to be routed by more than just the destination address.
Constraints:
------------
I want to make some form of this available in the 6.x tree
(and by extension 7.x) , but FreeBSD in general needs it so I might as
well do it in -current and back port the portions I need.
One of the ways that this can be done is to have the ability to
instantiate multiple kernel routing tables (which I will now
refer to as "Forwarding Information Bases" or "FIBs" for political
correctness reasons). Which FIB a particular packet uses to make
the next hop decision can be decided by a number of mechanisms.
The policies these mechanisms implement are the "Policies" referred
to in "Policy based routing".
One of the constraints I have if I try to back port this work to
6.x is that it must be implemented as a EXTENSION to the existing
ABIs in 6.x so that third party applications do not need to be
recompiled in timespan of the branch.
This first version will not have some of the bells and whistles that
will come with later versions. It will, for example, be limited to 16
tables in the first commit.
Implementation method, Compatible version. (part 1)
-------------------------------
For this reason I have implemented a "sufficient subset" of a
multiple routing table solution in Perforce, and back-ported it
to 6.x. (also in Perforce though not always caught up with what I
have done in -current/P4). The subset allows a number of FIBs
to be defined at compile time (8 is sufficient for my purposes in 6.x)
and implements the changes needed to allow IPV4 to use them. I have not
done the changes for ipv6 simply because I do not need it, and I do not
have enough knowledge of ipv6 (e.g. neighbor discovery) needed to do it.
Other protocol families are left untouched and should there be
users with proprietary protocol families, they should continue to work
and be oblivious to the existence of the extra FIBs.
To understand how this is done, one must know that the current FIB
code starts everything off with a single dimensional array of
pointers to FIB head structures (One per protocol family), each of
which in turn points to the trie of routes available to that family.
The basic change in the ABI compatible version of the change is to
extent that array to be a 2 dimensional array, so that
instead of protocol family X looking at rt_tables[X] for the
table it needs, it looks at rt_tables[Y][X] when for all
protocol families except ipv4 Y is always 0.
Code that is unaware of the change always just sees the first row
of the table, which of course looks just like the one dimensional
array that existed before.
The entry points rtrequest(), rtalloc(), rtalloc1(), rtalloc_ign()
are all maintained, but refer only to the first row of the array,
so that existing callers in proprietary protocols can continue to
do the "right thing".
Some new entry points are added, for the exclusive use of ipv4 code
called in_rtrequest(), in_rtalloc(), in_rtalloc1() and in_rtalloc_ign(),
which have an extra argument which refers the code to the correct row.
In addition, there are some new entry points (currently called
rtalloc_fib() and friends) that check the Address family being
looked up and call either rtalloc() (and friends) if the protocol
is not IPv4 forcing the action to row 0 or to the appropriate row
if it IS IPv4 (and that info is available). These are for calling
from code that is not specific to any particular protocol. The way
these are implemented would change in the non ABI preserving code
to be added later.
One feature of the first version of the code is that for ipv4,
the interface routes show up automatically on all the FIBs, so
that no matter what FIB you select you always have the basic
direct attached hosts available to you. (rtinit() does this
automatically).
You CAN delete an interface route from one FIB should you want
to but by default it's there. ARP information is also available
in each FIB. It's assumed that the same machine would have the
same MAC address, regardless of which FIB you are using to get
to it.
This brings us as to how the correct FIB is selected for an outgoing
IPV4 packet.
Firstly, all packets have a FIB associated with them. if nothing
has been done to change it, it will be FIB 0. The FIB is changed
in the following ways.
Packets fall into one of a number of classes.
1/ locally generated packets, coming from a socket/PCB.
Such packets select a FIB from a number associated with the
socket/PCB. This in turn is inherited from the process,
but can be changed by a socket option. The process in turn
inherits it on fork. I have written a utility call setfib
that acts a bit like nice..
setfib -3 ping target.example.com # will use fib 3 for ping.
It is an obvious extension to make it a property of a jail
but I have not done so. It can be achieved by combining the setfib and
jail commands.
2/ packets received on an interface for forwarding.
By default these packets would use table 0,
(or possibly a number settable in a sysctl(not yet)).
but prior to routing the firewall can inspect them (see below).
(possibly in the future you may be able to associate a FIB
with packets received on an interface.. An ifconfig arg, but not yet.)
3/ packets inspected by a packet classifier, which can arbitrarily
associate a fib with it on a packet by packet basis.
A fib assigned to a packet by a packet classifier
(such as ipfw) would over-ride a fib associated by
a more default source. (such as cases 1 or 2).
4/ a tcp listen socket associated with a fib will generate
accept sockets that are associated with that same fib.
5/ Packets generated in response to some other packet (e.g. reset
or icmp packets). These should use the FIB associated with the
packet being reponded to.
6/ Packets generated during encapsulation.
gif, tun and other tunnel interfaces will encapsulate using the FIB
that was in effect withthe proces that set up the tunnel.
thus setfib 1 ifconfig gif0 [tunnel instructions]
will set the fib for the tunnel to use to be fib 1.
Routing messages would be associated with their
process, and thus select one FIB or another.
messages from the kernel would be associated with the fib they
refer to and would only be received by a routing socket associated
with that fib. (not yet implemented)
In addition Netstat has been edited to be able to cope with the
fact that the array is now 2 dimensional. (It looks in system
memory using libkvm (!)). Old versions of netstat see only the first FIB.
In addition two sysctls are added to give:
a) the number of FIBs compiled in (active)
b) the default FIB of the calling process.
Early testing experience:
-------------------------
Basically our (IronPort's) appliance does this functionality already
using ipfw fwd but that method has some drawbacks.
For example,
It can't fully simulate a routing table because it can't influence the
socket's choice of local address when a connect() is done.
Testing during the generating of these changes has been
remarkably smooth so far. Multiple tables have co-existed
with no notable side effects, and packets have been routes
accordingly.
ipfw has grown 2 new keywords:
setfib N ip from anay to any
count ip from any to any fib N
In pf there seems to be a requirement to be able to give symbolic names to the
fibs but I do not have that capacity. I am not sure if it is required.
SCTP has interestingly enough built in support for this, called VRFs
in Cisco parlance. it will be interesting to see how that handles it
when it suddenly actually does something.
Where to next:
--------------------
After committing the ABI compatible version and MFCing it, I'd
like to proceed in a forward direction in -current. this will
result in some roto-tilling in the routing code.
Firstly: the current code's idea of having a separate tree per
protocol family, all of the same format, and pointed to by the
1 dimensional array is a bit silly. Especially when one considers that
there is code that makes assumptions about every protocol having the
same internal structures there. Some protocols don't WANT that
sort of structure. (for example the whole idea of a netmask is foreign
to appletalk). This needs to be made opaque to the external code.
My suggested first change is to add routing method pointers to the
'domain' structure, along with information pointing the data.
instead of having an array of pointers to uniform structures,
there would be an array pointing to the 'domain' structures
for each protocol address domain (protocol family),
and the methods this reached would be called. The methods would have
an argument that gives FIB number, but the protocol would be free
to ignore it.
When the ABI can be changed it raises the possibilty of the
addition of a fib entry into the "struct route". Currently,
the structure contains the sockaddr of the desination, and the resulting
fib entry. To make this work fully, one could add a fib number
so that given an address and a fib, one can find the third element, the
fib entry.
Interaction with the ARP layer/ LL layer would need to be
revisited as well. Qing Li has been working on this already.
This work was sponsored by Ironport Systems/Cisco
Reviewed by: several including rwatson, bz and mlair (parts each)
Obtained from: Ironport systems/Cisco
header is now in two parts: bsdxml.h and bsdxml_external.h, representing
the expat.h and expat_external.h headers. Updated the info on the man
page as well. Also, fixed a type-error in a printf in
sbin/ifconfig/regdomain.c that would cause a compiler warning.
Approved by: sam, phk
alowing the DDB output capture buffer to be easily extracted from
user space. Both of these commands include -M/-N arguments, allowing
them to be used with kernel crash dumps (or /dev/mem).
This makes it easier to use DDB scripting and output capture with
minidumps or full dumps rather than with text dumps, allowing DDB
output (scripted or otherwise) to be easily extracted from a crash
dump.
MFC after: 1 week
Discussed with: brooks, jhb
Note this includes changes to all drivers and moves some device firmware
loading to use firmware(9) and a separate module (e.g. ral). Also there
no longer are separate wlan_scan* modules; this functionality is now
bundled into the wlan module.
Supported by: Hobnob and Marvell
Reviewed by: many
Obtained from: Atheros (some bits)
