when I changed the allocator bits. This implements per-CPU mbtypes
stats by keeping net number of decrements/increments of a given mbtype
per-CPU and then summing all of the per-CPU mbtypes to produce the total
net number of allocated mbufs of the given mbtype.
Counters are carefully balanced to avoid/prevent underflows/overflows.
mbtypes stats are re-enabled with the idea that we may occasionally
(although very rarely) observe slight inconsistencies in the stat
reporting. Most of the time, we should be fine, though.
Also make appropriate modifications to netstat(1) and systat(1) to do
the necessary reporting.
Submitted by: Jiangyi Liu <jyliu@163.net>
The compatibility glue is still provided.
(This change is not yet reflected in the manpage, nor
in usage(). This will be fixed at a later time today,
with the general manpage cleanup commit.)
approximately the amount of memory allocated from the mbuf maps
and sitting in the mbuf allocator's cache containers, and display
in parantheses the percentage of said memory that is actually
in use at the given time `netstat -m' is executed.
Suggested by: mjacob
were indices in a dense array. The cpuids are a sparse set and treat
them as such, setting up containers only for CPUs activated during
mb_init().
- Fix netstat(1) and systat(1) to treat the per-CPU stats area as a sparse
map, in accordance with the above.
This allows us to properly boot with certain CPUs disactivated. However, if
we later decide to re-activate said CPUs, we will barf until we decide to
implement CPU spinon/spinoff callback hooks to allow for said CPUs' per-CPU
containers to get configured on their activation.
Reported by: mjacob
Partially (sys/ diffs) Submitted by: mjacob
- fix the problem that netstat doesn't show raw6 and icmp6 pcblist.
- make netstat use sysctl to retreive stats of ipv6 and icmpv6
instead of kread.
Obtained from: KAME
MFC after: 1 week
introduce a modified allocation mechanism for mbufs and mbuf clusters; one
which can scale under SMP and which offers the possibility of resource
reclamation to be implemented in the future. Notable advantages:
o Reduce contention for SMP by offering per-CPU pools and locks.
o Better use of data cache due to per-CPU pools.
o Much less code cache pollution due to excessively large allocation macros.
o Framework for `grouping' objects from same page together so as to be able
to possibly free wired-down pages back to the system if they are no longer
needed by the network stacks.
Additional things changed with this addition:
- Moved some mbuf specific declarations and initializations from
sys/conf/param.c into mbuf-specific code where they belong.
- m_getclr() has been renamed to m_get_clrd() because the old name is really
confusing. m_getclr() HAS been preserved though and is defined to the new
name. No tree sweep has been done "to change the interface," as the old
name will continue to be supported and is not depracated. The change was
merely done because m_getclr() sounds too much like "m_get a cluster."
- TEMPORARILY disabled mbtypes statistics displaying in netstat(1) and
systat(1) (see TODO below).
- Fixed systat(1) to display number of "free mbufs" based on new per-CPU
stat structures.
- Fixed netstat(1) to display new per-CPU stats based on sysctl-exported
per-CPU stat structures. All infos are fetched via sysctl.
TODO (in order of priority):
- Re-enable mbtypes statistics in both netstat(1) and systat(1) after
introducing an SMP friendly way to collect the mbtypes stats under the
already introduced per-CPU locks (i.e. hopefully don't use atomic() - it
seems too costly for a mere stat update, especially when other locks are
already present).
- Optionally have systat(1) display not only "total free mbufs" but also
"total free mbufs per CPU pool."
- Fix minor length-fetching issues in netstat(1) related to recently
re-enabled option to read mbuf stats from a core file.
- Move reference counters at least for mbuf clusters into an unused portion
of the cluster itself, to save space and need to allocate a counter.
- Look into introducing resource freeing possibly from a kproc.
Reviewed by (in parts): jlemon, jake, silby, terry
Tested by: jlemon (Intel & Alpha), mjacob (Intel & Alpha)
Preliminary performance measurements: jlemon (and me, obviously)
URL: http://people.freebsd.org/~bmilekic/mb_alloc/
Removed the ambiguity in -s, -f, -p and -i flags handling.
Basically, there are four displays (except others):
1. PCB display.
2. Protocol statistics display. (-s)
3. Interface statistics display. (-i)
4. Per-interface protocol statistics display. (-i -s)
All of the above except 3) can be limited to a particular
protocol family (-f) or a single protocol (-p).
Some examples:
1. netstat -f inet -- show PCBs of all INET protocols
2. netstat -p udp -- show PCB of UDP protocol only (NEW!)
3. netstat -s -- show protocol statistics for all families
4. netstat -s -f inet -- show INET protocols statistics
5. netstat -s -p icmp -- show ICMP protocol statistics
This is a work in progress. Manpage has been fixed slightly,
but is still incomplete.
This work was based on kame-20010528-freebsd43-snap.tgz and some
critical problem after the snap was out were fixed.
There are many many changes since last KAME merge.
TODO:
- The definitions of SADB_* in sys/net/pfkeyv2.h are still different
from RFC2407/IANA assignment because of binary compatibility
issue. It should be fixed under 5-CURRENT.
- ip6po_m member of struct ip6_pktopts is no longer used. But, it
is still there because of binary compatibility issue. It should
be removed under 5-CURRENT.
Reviewed by: itojun
Obtained from: KAME
MFC after: 3 weeks
Always print at least 3 bytes for IN_CLASSC_NET networks.
The standard 193.0.0 class C network for example, will now
be displayed as "193.0.0" as opposed to the confusing 193.
PR: bin/21546
MFC after: 1 week
A route generated from an RTF_CLONING route had the RTF_WASCLONED flag
set but did not have a reference to the parent route, as documented in
the rtentry(9) manpage. This prevented such routes from being deleted
when their parent route is deleted.
Now, for example, if you delete an IP address from a network interface,
all ARP entries that were cloned from this interface route are flushed.
This also has an impact on netstat(1) output. Previously, dynamically
created ARP cache entries (RTF_STATIC flag is unset) were displayed as
part of the routing table display (-r). Now, they are only printed if
the -a option is given.
netinet/in.c, netinet/in_rmx.c:
When address is removed from an interface, also delete all routes that
point to this interface and address. Previously, for example, if you
changed the address on an interface, outgoing IP datagrams might still
use the old address. The only solution was to delete and re-add some
routes. (The problem is easily observed with the route(8) command.)
Note, that if the socket was already bound to the local address before
this address is removed, new datagrams generated from this socket will
still be sent from the old address.
PR: kern/20785, kern/21914
Reviewed by: wollman (the idea)
statistics on a per network address basis.
Teach the IPv4 and IPv6 input/output routines to log packets/bytes
against the network address connected to the flow.
Teach netstat to display the per-address stats for IP protocols
when 'netstat -i' is evoked, instead of displaying the per-interface
stats.
that should be better.
The old code counted references to mbuf clusters by using the offset
of the cluster from the start of memory allocated for mbufs and
clusters as an index into an array of chars, which did the reference
counting. If the external storage was not a cluster then reference
counting had to be done by the code using that external storage.
NetBSD's system of linked lists of mbufs was cosidered, but Alfred
felt it would have locking issues when the kernel was made more
SMP friendly.
The system implimented uses a pool of unions to track external
storage. The union contains an int for counting the references and
a pointer for forming a free list. The reference counts are
incremented and decremented atomically and so should be SMP friendly.
This system can track reference counts for any sort of external
storage.
Access to the reference counting stuff is now through macros defined
in mbuf.h, so it should be easier to make changes to the system in
the future.
The possibility of storing the reference count in one of the
referencing mbufs was considered, but was rejected 'cos it would
often leave extra mbufs allocated. Storing the reference count in
the cluster was also considered, but because the external storage
may not be a cluster this isn't an option.
The size of the pool of reference counters is available in the
stats provided by "netstat -m".
PR: 19866
Submitted by: Bosko Milekic <bmilekic@dsuper.net>
Reviewed by: alfred (glanced at by others on -net)
There was a missing description for a new flags to netstat.
I already added the fix to netstat man, but usage() change is
also necessary.
Specified by: Ben Smithurst <ben@scientia.demon.co.uk>
Approved by: jkh
kernel IPv6 multicast routing support.
pim6 dense mode daemon
pim6 sparse mode daemon
netstat support of IPv6 multicast routing statistics
Merging to the current and testing with other existing multicast routers
is done by Tatsuya Jinmei <jinmei@kame.net>, who writes and maintainances
the base code in KAME distribution.
Make world check and kernel build check was also successful.
only when either of sflag and "-f inet6" is specified.
-fix the indentation of default output
Specified by: Stephen McKay <syssgm@detir.qld.gov.au>
Reviewed and Confirmed by: Stephen McKay <syssgm@detir.qld.gov.au>
information in 80 columns.
TODO: IPv6 related information is not likely to be kept in 80 columns, anyway.
Some more print modes could be added,
but what is the priority between those modes?
-print out all information even if they don't fit into 80 columns
-strip off some information to fit them into 80 columns
Reviewed by: markm
packet divert at kernel for IPv6/IPv4 translater daemon
This includes queue related patch submitted by jburkhol@home.com.
Submitted by: queue related patch from jburkhol@home.com
Reviewed by: freebsd-arch, cvs-committers
Obtained from: KAME project
Been in production for 3 years now. Gives Instant Frame relay to if_sr
and if_ar drivers, and PPPOE support soon. See:
ftp://ftp.whistle.com/pub/archie/netgraph/index.html
for on-line manual pages.
Reviewed by: Doug Rabson (dfr@freebsd.org)
Obtained from: Whistle CVS tree
track.
The $Id$ line is normally at the bottom of the main comment block in the
man page, separated from the rest of the manpage by an empty comment,
like so;
.\" $Id$
.\"
If the immediately preceding comment is a @(#) format ID marker than the
the $Id$ will line up underneath it with no intervening blank lines.
Otherwise, an additional blank line is inserted.
Approved by: bde
the kernel; this was left over from the earlier protocol-dependent
kernel multicast routing code.
Learn how to handle the malloc'd multicast routing table (instead of
expecting it to be in mbufs)
Didn't fix the alignment of the output fields on alpha where addresses
require 16 characters to print.
Added a dummy field to the pt_u union to help the alpha compiler align
the u_sa field in a suiable way.
name for AF_LINK routing entries. This makes debugging
network problems more difficult.
PR: 4182
Reviewed by: phk
Submitted by: Craig Leres <leres@ee.lbl.gov>
convert icmp_stats() to use sysctl(3) to retrieve the information
rather than kvm. This makes it easy to also print whether ICMP address
mask responses are enabled, so do so.
Remove the dns lookup code in the ipx functions. That is bogus and slows
things like netstat -r(f ipx) down, without gaining anything.
Remove the ipx error protocol statistics.
the kernel malloc, netstat was never updated to reflect the fact that
there are once again allocated-but-free mbufs, just as there are
clusters, and so the information presented about how much memory was
allocated to the network was bogus. Fixed.
This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.
Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.
Tor Egge reports counter wrap and requests and update to quad_t sized
counters, which is also a good thing to do, but I'm unhappy about adding
two more instructions into the code path every time we doink a counter.
Maybe with or after the Lite2 merge...
Submitted by: Tor Egge <Tor.Egge@idt.ntnu.no>
do it themselves. (Some of these programs actually depended on this
beyond compiling the definition of struct ifinfo!) Also fix up some
other #include messes while we're at it.
This stuff should not be too destructive if the IPDIVERT is not compiled in..
be aware that this changes the size of the ip_fw struct
so ipfw needs to be recompiled to use it.. more changes coming to clean this up.
messy 130 column collage, output the system totals -or- info for a
specific interface if -I is given. Also wait for <interval> before
outputting the first sample so that it represents meaningful data (as
opposed to the total since the system was booted - most busy systems
wrap around many times during their operation, so these numbers are
only misleading).
route.c: look up the netname as-is first before the shifted name. this
makes a big speed difference, as the lookups are generally local
DNS. The shifted names can be very wrong (there is still guessing
and fudging involved) and usually go remote, taking a long time
to fail. If you have the RFC reccomended netnames in your reverse
lookups, this is even faster still.
main.c: dont do a sethostent(1) - this is causing the resolver to use a
VC (tcp) connection to the resolver, which has more overheads and
is slower than the default UDP case. This once made sense when
everything was based on text host tables.
patches to merge the two IPX packages to work with each other and to
not break make-world :)
IPXrouted should be working now, (or at least compiling) :)
Submitted by: Mike Mitchell, supervisor@alb.asctmd.com
This is a bulk mport of Mike's IPX/SPX protocol stacks and all the
related gunf that goes with it..
it is not guaranteed to work 100% correctly at this time
but as we had several people trying to work on it
I figured it would be better to get it checked in so
they could all get teh same thing to work on..
Mikes been using it for a year or so
but on 2.0
more changes and stuff will be merged in from other developers now that this is in.
Mike Mitchell, Network Engineer
AMTECH Systems Corporation, Technology and Manufacturing
8600 Jefferson Street, Albuquerque, New Mexico 87113 (505) 856-8000
supervisor@alb.asctmd.com
the way it stores and handles "interface". The previous behavior resulted
in strange output from 'w' and 'ps' when an interface specification was
given to netstat.
Look at error return of kread() and stop on error.
Fix warning in kread() to not output "kvm_read:" twice.
Killed PCB cache misses stat as we no longer have it.
