d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
f7574a2276
freebsd-skq/sys/kern
History
bmilekic f7574a2276 Bring in mbuma to replace mballoc. 
mbuma is an Mbuf & Cluster allocator built on top of a number of
extensions to the UMA framework, all included herein.

Extensions to UMA worth noting:
  - Better layering between slab <-> zone caches; introduce
    Keg structure which splits off slab cache away from the
    zone structure and allows multiple zones to be stacked
    on top of a single Keg (single type of slab cache);
    perhaps we should look into defining a subset API on
    top of the Keg for special use by malloc(9),
    for example.
  - UMA_ZONE_REFCNT zones can now be added, and reference
    counters automagically allocated for them within the end
    of the associated slab structures.  uma_find_refcnt()
    does a kextract to fetch the slab struct reference from
    the underlying page, and lookup the corresponding refcnt.

mbuma things worth noting:
  - integrates mbuf & cluster allocations with extended UMA
    and provides caches for commonly-allocated items; defines
    several zones (two primary, one secondary) and two kegs.
  - change up certain code paths that always used to do:
    m_get() + m_clget() to instead just use m_getcl() and
    try to take advantage of the newly defined secondary
    Packet zone.
  - netstat(1) and systat(1) quickly hacked up to do basic
    stat reporting but additional stats work needs to be
    done once some other details within UMA have been taken
    care of and it becomes clearer to how stats will work
    within the modified framework.

From the user perspective, one implication is that the
NMBCLUSTERS compile-time option is no longer used.  The
maximum number of clusters is still capped off according
to maxusers, but it can be made unlimited by setting
the kern.ipc.nmbclusters boot-time tunable to zero.
Work should be done to write an appropriate sysctl
handler allowing dynamic tuning of kern.ipc.nmbclusters
at runtime.

Additional things worth noting/known issues (READ):
   - One report of 'ips' (ServeRAID) driver acting really
     slow in conjunction with mbuma.  Need more data.
     Latest report is that ips is equally sucking with
     and without mbuma.
   - Giant leak in NFS code sometimes occurs, can't
     reproduce but currently analyzing; brueffer is
     able to reproduce but THIS IS NOT an mbuma-specific
     problem and currently occurs even WITHOUT mbuma.
   - Issues in network locking: there is at least one
     code path in the rip code where one or more locks
     are acquired and we end up in m_prepend() with
     M_WAITOK, which causes WITNESS to whine from within
     UMA.  Current temporary solution: force all UMA
     allocations to be M_NOWAIT from within UMA for now
     to avoid deadlocks unless WITNESS is defined and we
     can determine with certainty that we're not holding
     any locks when we're M_WAITOK.
   - I've seen at least one weird socketbuffer empty-but-
     mbuf-still-attached panic.  I don't believe this
     to be related to mbuma but please keep your eyes
     open, turn on debugging, and capture crash dumps.

This change removes more code than it adds.

A paper is available detailing the change and considering
various performance issues, it was presented at BSDCan2004:
http://www.unixdaemons.com/~bmilekic/netbuf_bmilekic.pdf
Please read the paper for Future Work and implementation
details, as well as credits.

Testing and Debugging:
    rwatson,
    brueffer,
    Ketrien I. Saihr-Kesenchedra,
    ...
Reviewed by: Lots of people (for different parts)
2004-05-31 21:46:06 +00:00
..
bus_if.m Introduce BUS_CONFIG_INTR(). The method allows devices to tell parents 2003-09-10 21:37:10 +00:00
clock_if.m
device_if.m
genassym.sh
imgact_aout.c Locking for the per-process resource limits structure. 2004-02-04 21:52:57 +00:00
imgact_elf32.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
imgact_elf64.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
imgact_elf.c Utilize sf_buf_alloc() rather than pmap_qenter() (and sometimes 2004-04-23 03:01:40 +00:00
imgact_gzip.c Locking for the per-process resource limits structure. 2004-02-04 21:52:57 +00:00
imgact_shell.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
inflate.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
init_main.c Loudly announce WITNESS and DIAGNOSTIC options and warn about reduced 2004-02-29 16:56:54 +00:00
init_sysent.c Regen. 2004-04-05 10:17:23 +00:00
kern_acct.c Fixed some style bugs (mainly misplaced comments, and totally disordered 2004-03-04 09:47:09 +00:00
kern_acl.c Update my personal copyrights and NETA copyrights in the kernel 2004-02-22 00:33:12 +00:00
kern_alq.c Allow MAC policies to block/revoke kern_alq write access to a file. 2003-10-25 16:10:41 +00:00
kern_clock.c Fix a race condition which could result in profprocs being decremented 2004-05-03 00:48:11 +00:00
kern_condvar.c Associate a simple count of waiters with each condition variable. The 2004-04-06 19:17:46 +00:00
kern_conf.c Correctly account for extra bits in unit numbers when looking for 2004-03-11 14:11:02 +00:00
kern_context.c Change the clear_ret argument of get_mcontext() to be a flags argument. 2003-11-09 20:31:04 +00:00
kern_descrip.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_environment.c If the buffer supplied to kenv(KENV_DUMP, ...) isn't big enough, 2004-04-28 01:27:33 +00:00
kern_event.c Fix filt_timer* races: Finish initializing a knote before we pass it to 2004-04-07 05:59:57 +00:00
kern_exec.c Clear KSE thread flags after KSE thread mode is ended. The side effect 2004-05-21 14:50:23 +00:00
kern_exit.c Retire cpu_sched_exit(); it is not used any more. 2004-05-26 12:09:39 +00:00
kern_fork.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_idle.c Always set a process' state to normal when it is fully constructed in 2004-02-05 21:01:37 +00:00
kern_intr.c Fixed some style bugs in previous commit (mainly an insertion sort error 2004-04-17 02:46:05 +00:00
kern_jail.c Fix sysctl name: security.jail.getfsstate_getfsstatroot_only -> 2004-05-20 05:28:44 +00:00
kern_kse.c Clear KSE thread flags after KSE thread mode is ended. The side effect 2004-05-21 14:50:23 +00:00
kern_kthread.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
kern_ktr.c Update the license on this file to be a bit more sane. 2003-09-10 01:09:32 +00:00
kern_ktrace.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_linker.c Since we go to the trouble of compiling the kobj ops table for each class, 2004-05-17 21:24:39 +00:00
kern_lock.c Add pid to the info printed in lockmgr_printinfo. This makes VFS 2004-01-06 04:34:13 +00:00
kern_lockf.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_mac.c Add /* !MAC */ to final #endif. 2004-05-03 22:54:46 +00:00
kern_malloc.c Bring in mbuma to replace mballoc. 2004-05-31 21:46:06 +00:00
kern_mbuf.c Bring in mbuma to replace mballoc. 2004-05-31 21:46:06 +00:00
kern_mib.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_module.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
kern_mtxpool.c Rearrange the SYSINIT order to call lockmgr_init() earlier so that 2003-07-16 01:00:39 +00:00
kern_mutex.c Add a new kernel option MUTEX_WAKE_ALL that changes the mutex unlock code 2004-04-06 19:12:24 +00:00
kern_ntptime.c Annual NTP kernel code spring-cleaning: 2004-03-14 15:23:05 +00:00
kern_physio.c Send B_PHYS out to pasture, it no longer serves any function. 2003-11-15 09:28:09 +00:00
kern_poll.c Ensure that the poll_burst <= poll_burst_max constraint really holds. 2004-04-15 07:38:44 +00:00
kern_proc.c Implement the new KERN_PROC_RGID option, and also implement the 2004-05-22 23:11:44 +00:00
kern_prot.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_resource.c Fix rtprio() to do sensible things when called from threaded processes. 2004-05-08 08:56:05 +00:00
kern_sema.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
kern_shutdown.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_sig.c Fixed some style bugs in tdsigwakeup(). 2004-05-21 10:02:24 +00:00
kern_subr.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_switch.c - style fixes to the critical_exit() KASSERT(). 2004-02-02 08:13:27 +00:00
kern_sx.c Fix _sx_assert() to panic() rather than printf() when an assertion fails 2004-02-27 16:13:44 +00:00
kern_synch.c Fixed some common printf format errors. Don't assume that "struct foo *" 2004-05-14 20:51:42 +00:00
kern_syscalls.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
kern_sysctl.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_tc.c Just because the timecounter reads the same value on two samples 2004-03-04 14:14:23 +00:00
kern_thr.c If you're trying to find out if a thread is valid and in 2004-04-19 14:20:01 +00:00
kern_thread.c Clear KSE thread flags after KSE thread mode is ended. The side effect 2004-05-21 14:50:23 +00:00
kern_time.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
kern_timeout.c The paper "Hashed Timers and Hierarchical Wheels: Data Structures for the 2004-04-25 04:10:17 +00:00
kern_umtx.c Use the proc lock to sleep on a libthr umtx. 2004-03-27 14:32:03 +00:00
kern_uuid.c Fix generation of random multicast MAC address. 2004-01-22 13:34:11 +00:00
kern_xxx.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
ksched.c sched_setscheduler: Return EINVAL when a invalid policy is specified, 2003-09-13 18:46:24 +00:00
link_elf_obj.c If a symbol has section+offset definitions provided, always use instead 2004-05-18 05:15:43 +00:00
link_elf.c Include <sys/gmon.h> instead of <machine/profile.h> for the declaration 2004-05-19 14:36:38 +00:00
linker_if.m The method link_preload_finish is not static. 2003-09-20 17:39:32 +00:00
Make.tags.inc Finish driving a stake through the heart of netns and the associated 2003-03-05 19:24:24 +00:00
Makefile Finish driving a stake through the heart of netns and the associated 2003-03-05 19:24:24 +00:00
makesyscalls.sh Remove the namespace column from the syscalls tables. We don't actually 2003-12-23 03:50:43 +00:00
md4c.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
md5c.c Revert stuff which accidentally ended up in the previous commit. 2003-07-22 10:36:36 +00:00
p1003_1b.c Use __FBSDID(). 2003-06-11 06:34:30 +00:00
posix4_mib.c Use __FBSDID(). 2003-06-11 06:34:30 +00:00
sched_4bsd.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
sched_ule.c There was a thread on "unusually high load averages" when running under 2004-04-22 21:37:46 +00:00
subr_acl_posix1e.c Update my personal copyrights and NETA copyrights in the kernel 2004-02-22 00:33:12 +00:00
subr_autoconf.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
subr_blist.c Expand inline the relevant parts of src/COPYRIGHT for Matt Dillon's 2003-08-12 23:24:05 +00:00
subr_bus.c Sysctl hw.bus.devctl_disable shouldn't be writtable from inside a jail. 2004-05-26 16:36:32 +00:00
subr_clist.c Revert stuff which accidentally ended up in the previous commit. 2003-07-22 10:36:36 +00:00
subr_clock.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
subr_devstat.c Device megapatch 4/6: 2004-02-21 21:10:55 +00:00
subr_disk.c Report bio_pblkbo instead of bio_blkno. 2003-10-18 17:27:10 +00:00
subr_eventhandler.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
subr_hints.c Add a resource_disabled() helper function that returns true (non-zero) if 2003-07-02 16:01:38 +00:00
subr_kobj.c * Add multiple inheritance to kobj. Each class can have zero or more base 2003-10-16 09:16:28 +00:00
subr_log.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
subr_mbpool.c Pass MTX_DEF as the last argument to mtx_init() instead of 0. This 2003-12-07 21:53:41 +00:00
subr_mchain.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
subr_module.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
subr_msgbuf.c Put the message about msgbuf cksum mismatch under bootverbose and tell 2003-09-05 11:12:00 +00:00
subr_param.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
subr_pcpu.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
subr_power.c Move the kernel power change printf under bootverbose since the 2004-01-02 18:24:13 +00:00
subr_prf.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
subr_prof.c Enable MI bits for gcc -ftest-coverage -fprofile-arcs on amd64. 2004-05-29 01:18:14 +00:00
subr_rman.c Fix off by one error, twice. 2004-04-12 23:02:21 +00:00
subr_rtc.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
subr_sbuf.c Mechanical whistespace cleanup. 2004-02-17 10:21:03 +00:00
subr_scanf.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
subr_sleepqueue.c Fixed some common printf format errors. Don't assume that "struct foo *" 2004-05-14 20:51:42 +00:00
subr_smp.c Move the CPU newbus attachment to i386 legacy. The acpi_cpu device will 2004-05-06 15:54:02 +00:00
subr_taskqueue.c Tidy up the thread taskqueue implementation and close a lost wakeup race. 2004-02-19 22:03:52 +00:00
subr_trap.c Remove unused variable. 2004-03-31 08:20:44 +00:00
subr_turnstile.c Rename turnstile_wakeup() to turnstile_broadcast() to make the naming 2004-04-06 19:07:21 +00:00
subr_witness.c Emit a traceback when witness_trace is set and witness_warn() is 2004-03-23 00:32:27 +00:00
sys_generic.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
sys_pipe.c Revise the direct or optimized case to use uiomove_fromphys() by the reader 2004-03-27 19:50:23 +00:00
sys_process.c Finish fixing up Alpha to work with an MP safe ptrace(): 2004-04-01 20:56:44 +00:00
sys_socket.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
syscalls.c Regen. 2004-04-05 10:17:23 +00:00
syscalls.master Add lgetfh(2) which is like getfh(2) but doesn't follow symlinks. 2004-04-05 10:15:53 +00:00
sysv_ipc.c Use __FBSDID(). 2003-06-11 00:56:59 +00:00
sysv_msg.c Add missing #include <sys/module.h> 2004-05-30 20:34:58 +00:00
sysv_sem.c Add missing #include <sys/module.h> 2004-05-30 20:34:58 +00:00
sysv_shm.c Add missing #include <sys/module.h> 2004-05-30 20:34:58 +00:00
tty_compat.c Compare t_brkc against (char)_POSIX_VDISABLE, not against -1. 2004-05-07 15:35:38 +00:00
tty_conf.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
tty_cons.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
tty_pty.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
tty_subr.c Revert stuff which accidentally ended up in the previous commit. 2003-07-22 10:36:36 +00:00
tty_tty.c Device megapatch 4/6: 2004-02-21 21:10:55 +00:00
tty.c Remove advertising clause from University of California Regent's 2004-04-07 20:46:16 +00:00
uipc_accf.c Add some missing <sys/module.h> includes which are masked by the 2004-05-30 17:57:46 +00:00
uipc_cow.c In some cases, sf_buf_alloc() should sleep with pri PCATCH; in others, it 2004-04-03 09:16:27 +00:00
uipc_domain.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
uipc_jumbo.c Lock the vm object when removing a page. 2003-06-11 21:23:04 +00:00
uipc_mbuf2.c Bring in mbuma to replace mballoc. 2004-05-31 21:46:06 +00:00
uipc_mbuf.c Bring in mbuma to replace mballoc. 2004-05-31 21:46:06 +00:00
uipc_proto.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
uipc_sem.c Add missing #include <sys/module.h> 2004-05-30 20:34:58 +00:00
uipc_sockbuf.c Bring in mbuma to replace mballoc. 2004-05-31 21:46:06 +00:00
uipc_socket2.c Bring in mbuma to replace mballoc. 2004-05-31 21:46:06 +00:00
uipc_socket.c Bring in mbuma to replace mballoc. 2004-05-31 21:46:06 +00:00
uipc_syscalls.c Bring in mbuma to replace mballoc. 2004-05-31 21:46:06 +00:00
uipc_usrreq.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
vfs_acl.c Update my personal copyrights and NETA copyrights in the kernel 2004-02-22 00:33:12 +00:00
vfs_aio.c Add missing #include <sys/module.h> 2004-05-30 20:34:58 +00:00
vfs_bio.c Avoid pointless zeroing of the bogus page in vfs_bio_clrbuf(). 2004-05-08 06:46:40 +00:00
vfs_cache.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
vfs_cluster.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
vfs_default.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
vfs_export.c Include <sys/mutex.h> and its prerequisite <sys/lock.h> instesd of depending 2004-04-21 12:10:30 +00:00
vfs_extattr.c Eliminate a memory leak in kern_symlink() that could occur if 2004-05-11 10:42:02 +00:00
vfs_init.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
vfs_lookup.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
vfs_mount.c Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00
vfs_subr.c Assert Giant in vrele(). 2004-05-31 19:06:01 +00:00
vfs_syscalls.c Eliminate a memory leak in kern_symlink() that could occur if 2004-05-11 10:42:02 +00:00
vfs_vnops.c Assert Giant in vn_start_write() and vn_finished_write(). 2004-05-31 20:56:10 +00:00
vnode_if.src Remove advertising clause from University of California Regent's license, 2004-04-05 21:03:37 +00:00