Modify "netstat -mb" to use libmemstat(3) when acting on a live system,
with a number of positive benefits:
- Start using UMA(9) statistics for mbufs and clusters, which avoids
using the mbuf allocator statistics which suffer from races under
load on SMP. This should eliminate "negative" mbuf counts in
netstat -mb.
- We are now able to track cached (free) mbufs and clusters and count
it towards memory allocated by the network stack.
- We are now also able to track memory allocated to mbuf tags since
libmemstat(3) can also query malloc(9). We don't print this except
as part of the total (for now - #if 0).
- We are now able to track mbuf/cluster/packet allocation failures,
although they are not currently printed (#if 0).
- Don't print out sfbuf statistics when running on a kernel core, as
currently that code is able only to query sysctl for statistics.
mbuf.c:1.44:
Since libmemstat(3) now supports its own error management mechanism,
use that instead of trying to use errno, in order to produce a
sensible error message.
Approved by: re (kensmith)
If a retrieved UMA zone is a secondary zone, don't report keg free
items, as they actually belong to the primary zone, and maye otherwise
be reported more than once.
Approved by: re (kensmith)
Rename UMA_MAX_NAME to UTH_MAX_NAME, since it's a maximum in the
monitoring API, which might or might not be the same as the internal
maximum (currently none).
Export flag information on UMA zones -- in particular, whether or
not this is a secondary zone, and so the keg free count should be
considered in that light.
Approved by: re (kensmith)
Use an interrupt gate for the NMI handler and prevent too-early
enabling of interrupts inside of trap().
Revert rev 1.113 of "sys/i386/i386/exception.s" as it is no longer
needed.
Approved by: re (kensmith)
> Clear the PROMISC flag from the vlan interface when we remove a member. We
> checked for IFT_L2VLAN in bridge_ioctl_add() but not bridge_delete_member().
>
> Approved by: mlaier (mentor)
>
> Revision Changes Path
> 1.13 +1 -0 src/sys/net/if_bridge.c
Approved by: re (kensmith), mlaier (mentor)
1) An unquoted space is always a separator, even when not "in_arg".
2) When a new destination buffer must be allocated during variable
substitution, only copy data from the active buffer to the new
one when we *are* "in_arg".
3) Fix minor memory leak when expanding ${variable}s
Approved by: re(kensmith)
Add support for AVM BlueFRITZ! USB Bluetooth Adapter v2.0. It appears that
there are at least two versions of the adapter. Version 1 (product ID 0x2200)
of the adapter does not work with ng_ubt(4) and require special driver and
firmware. Version 2 (product ID 0x3800) seems to work just fine, except it
does not have bDeviceClass, bDeviceSubClass and bDeviceProtocol set to required
(by specification) values. This change forces ng_ubt(4) to attach to the
version 2 adapter.
Ignore AVM BlueFRITZ! USB Bluetooth Adapter v1.0 (product ID 0x2200).
It does not work with ng_ubt(4) and require special driver and firmware.
Submitted by: Rainer Goellner < rainer at jabbe dot de >
Approved by: re (kensmith)
Obtained from: Marcel Holtmann < marcel at holtmann dot org >
Remove duplicate initialization of mpo_create_stub pointer.
PR: 83779
Submitted by: Wojciech A. Koszek <dunstan at freebsd dot czest dot pl>
Approved by: re (kensmith)
RELENG_6:
Define four constants, MBUF_{,MEM,CLUSTER,PACKET,TAG}_MEM_NAME, which
are string names for their respective UMA zones and malloc types, and
are passed into uma_zcreate() and MALLOC_DEFINE(). Export them
outside of _KERNEL in mbuf.h so that netstat can reference them.
Change the names to improve consistency, with each zone/type
associated with the mbuf allocator being prefixed mbuf_.
Approved by: re (kensmith)
Add libmemstat(3), a library for use by debugging and monitoring
applications in tracking kernel memory statistics. It provides an
abstracted interface to uma(9) and malloc(9) statistics, wrapped
around the recently added binary stream sysctls for the allocators.
Using this interface, it is easy to build monitoring tools, query
specific memory types for usage information, etc. Facilities are
provided for binding caller-provided data to memory types,
incremental updates of memory types, and queries that span multiple
allocators.
Support for additional allocators is (relatively) easy to add.
The API for libmemstat(3) will probably change some over time as
consumers are written, and requirements evolve. It is written to
avoid encoding ABIs for data structure layout into consuming
applications for this reason.
This change merges all improvements, enhancements, bug fixes, etc, as
of this date. Revisions merged are:
Makefile:1.1-1.3
libmemstat.3:1.1-1.6
memstat.h:1.1-1.7
memstat_all.c:1.1
memstat_internal.h:1.5
memstat_malloc.c:1.1-1.4
memstat_uma.c:1.1-1.7
Approved by: re (kensmith)
Introduce a new sysctl, kern.malloc_stats, which exports kernel malloc
statistics via a binary structure stream:
- Add structure 'malloc_type_stream_header', which defines a stream
version, definition of MAXCPUS used in the stream, and a number of
malloc_type records in the stream.
- Add structure 'malloc_type_header', which defines the name of the
malloc type being reported on.
- When the sysctl is queried, return a stream header, followed by a
series of type descriptions, each consisting of a type header
followed by a series of MAXCPUS malloc_type_stats structures holding
per-CPU allocation information. Typical values of MAXCPUS will be 1
(UP compiled kernel) and 16 (SMP compiled kernel).
This query mechanism allows user space monitoring tools to extract
memory allocation statistics in a machine-readable form, and to do so
at a per-CPU granularity, allowing monitoring of allocation patterns
across CPUs in order to better understand the distribution of work and
memory flow over multiple CPUs.
While here:
- Bump statistics width to uint64_t, and hard code using fixed-width
type in order to be more sure about structure layout in the stream.
We allocate and free a lot of memory.
- Add kmemcount, a counter of the number of registered malloc types,
in order to avoid excessive manual counting of types. Export via a
new sysctl to allow user-space code to better size buffers.
- De-XXX comment on no longer maintaining the high watermark in old
sysctl monitoring code.
A follow-up commit of libmemstat(3), a library to monitor kernel memory
allocation, will occur in the next few days. Likewise, similar changes
to UMA.
kern_malloc.c:1.144:
Correct build on 64-bit: cast u_int64_t to (unsigned long long) before
printfing as (unsigned long long). 32-bit build on i386 didn't notice
this. Whoops.
Reported by: arved
Tested by: sledge
Approved by: re (kensmith)
Export pfsyncstats via sysctl "net.inet.pfsync" in order to print them
with netstat (seperate commit).
Requested by: glebius
Approved by: re (kensmith)
Move eventhandler for 'ifnet_departure_event' at the end of the progress.
Some of the (IPv6) cleanup functions send packets to inform peers of the
departure. These packets confused users of ifnet_departure_event (pf at
the moment).
PR: kern/80627
Tested by: Divacky Roman
Approved by: re (kensmith)
Further UMA statistics related changes:
- Add a new uma_zfree_internal() flag, ZFREE_STATFREE, which causes it to
to update the zone's uz_frees statistic. Previously, the statistic was
updated unconditionally.
- Use the flag in situations where a "real" free occurs: i.e., one where
the caller is freeing an allocated item, to be differentiated from
situations where uma_zfree_internal() is used to tear down the item
during slab teardown in order to invoke its fini() method. Also use
the flag when UMA is freeing its internal objects.
- When exchanging a bucket with the zone from the per-CPU cache when
freeing an item, flush cache statistics back to the zone (since the
zone lock and critical section are both held) to match the allocation
case.
Approved by: re (kensmith)
Use mp_maxid in preference to MAXCPU when creating exports of UMA
per-CPU cache statistics. UMA sizes the cache array based on the
number of CPUs at boot (mp_maxid + 1), and iterating based on MAXCPU
could read off the end of the array (into the next zone).
Reported by: yongari
Approved by: re (kensmith)
RELENG_6:
Improve canonicalization of copyrights. Order copyrights by order of
assertion (jeff, bmilekic, rwatson).
Suggested ages ago by: bde
Approved by: re (kensmith)
Move the unlocking of the zone mutex in sysctl_vm_zone_stats() so that
it covers the following of the uc_alloc/freebucket cache pointers.
Originally, I felt that the race wasn't helped by holding the mutex,
hence a comment in the code and not holding it across the cache access.
However, it does improve consistency, as while it doesn't prevent
bucket exchange, it does prevent bucket pointer invalidation. So a
race in gathering cache free space statistics still can occur, but not
one that follows an invalid bucket pointer, if the mutex is held.
Submitted by: yongari
Approved by: re (kensmith)
RELENG_6:
Track UMA(9) allocation failures by zone, and export via sysctl.
Requested by: victor cruceru <victor dot cruceru at gmail dot com>
Approved by: re (kensmith)
RELENG_6:
Introduce a new sysctl, vm.zone_stats, which exports UMA(9) allocator
statistics via a binary structure stream:
- Add structure 'uma_stream_header', which defines a stream version,
definition of MAXCPUs used in the stream, and the number of zone
records in the stream.
- Add structure 'uma_type_header', which defines the name, alignment,
size, resource allocation limits, current pages allocated, preferred
bucket size, and central zone + keg statistics.
- Add structure 'uma_percpu_stat', which, for each per-CPU cache,
includes the number of allocations and frees, as well as the number
of free items in the cache.
- When the sysctl is queried, return a stream header, followed by a
series of type descriptions, each consisting of a type header
followed by a series of MAXCPUs uma_percpu_stat structures holding
per-CPU allocation information. Typical values of MAXCPU will be
1 (UP compiled kernel) and 16 (SMP compiled kernel).
This query mechanism allows user space monitoring tools to extract
memory allocation statistics in a machine-readable form, and to do so
at a per-CPU granularity, allowing monitoring of allocation patterns
across CPUs in order to better understand the distribution of work and
memory flow over multiple CPUs.
While here, also export the number of UMA zones as a sysctl
vm.uma_count, in order to assist in sizing user swpace buffers to
receive the stream.
A follow-up commit of libmemstat(3), a library to monitor kernel memory
allocation, will occur in the next few days. This change directly
supports converting netstat(1)'s "-mb" mode to using UMA-sourced stats
rather than separately maintained mbuf allocator statistics.
Approved by: re (kensmith)
In an earlier world order, UMA would flush per-CPU statistics to the
zone whenever it was moving buckets between the zone and the cache,
or when coalescing statistics across the CPU. Remove flushing of
statistics to the zone when coalescing statistics as part of sysctl,
as we won't be running on the right CPU to write to the cache
statistics.
Add a missed gathering of statistics: when uma_zalloc_internal()
does a special case allocation of a single item, make sure to update
the zone statistics to represent this. Previously this case wasn't
accounted for in user-visible statistics.
Approved by: re (kensmith)
from struct addrinfo. This change break ABI compatibility on
64 bit arch.
include/netdb.h: 1.39
lib/libc/net/getaddrinfo.c: 1.70
Approved by: re (kensmith)
