d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
76e63232b6
freebsd-dev/sys/kern
History
Gleb Smirnoff 1fbe6a82f4 Improve reference counting of EXT_SFBUF pages attached to mbufs. 
o Do not use UMA refcount zone. The problem with this zone is that
  several refcounting words (16 on amd64) share the same cache line,
  and issueing atomic(9) updates on them creates cache line contention.
  Also, allocating and freeing them is extra CPU cycles.
  Instead, refcount the page directly via vm_page_wire() and the sfbuf
  via sf_buf_alloc(sf_buf_page(sf)) [1].

o Call refcounting/freeing function for EXT_SFBUF via direct function
  call, instead of function pointer. This removes barrier for CPU
  branch predictor.

o Do not cleanup the mbuf to be freed in mb_free_ext(), merely to
  satisfy assertion in mb_dtor_mbuf(). Remove the assertion from
  mb_dtor_mbuf(). Use bcopy() instead of manual assignments to
  copy m_ext in mb_dupcl().

[1] This has some problems for now. Using sf_buf_alloc() merely to
    increase refcount is expensive, and is broken on sparc64. To be
    fixed.

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2014-07-11 19:40:50 +00:00
..
bus_if.m Add a BUS_CHILD_DELETED() method that a bus can hook to allow it to cleanup 2012-08-21 18:13:09 +00:00
capabilities.conf Allow sigwait(2) in capabilities mode. 2014-01-28 01:49:49 +00:00
clock_if.m
cpufreq_if.m
device_if.m Revert r239178 and implement two new functions, namely 2012-08-15 15:42:57 +00:00
genassym.sh
imgact_aout.c Cosmetics: define FREEBSD32_MINUSER and AOUT32_MINUSER for struct 2012-07-22 13:41:45 +00:00
imgact_binmisc.c sys/kern/imgact_binmisc.c -- free the right pointer mask vs magic 2014-04-08 22:12:01 +00:00
imgact_elf32.c
imgact_elf64.c
imgact_elf.c Remove ia64. 2014-07-07 00:27:09 +00:00
imgact_gzip.c Add a mmap flag (MAP_32BIT) on 64-bit platforms to request that a mapping use 2013-09-09 18:11:59 +00:00
imgact_shell.c
inflate.c
init_main.c Rename global cnt to vm_cnt to avoid shadowing. 2014-03-22 10:26:09 +00:00
init_sysent.c Regen per r263318. 2014-03-18 21:34:11 +00:00
kern_acct.c acct: create a special plimit object and set it for exiting processes 2013-06-30 19:08:06 +00:00
kern_alq.c Prevent alq from panic when the invalid alq_file path specified. 2014-04-05 16:54:47 +00:00
kern_clock.c dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE 2013-11-26 08:46:27 +00:00
kern_clocksource.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_condvar.c Fix lc_lock/lc_unlock() support for rmlocks held in shared mode. With 2013-09-20 23:06:21 +00:00
kern_conf.c Remove global device lock acquisition from dev_relthread(), replacing it 2013-10-22 10:40:26 +00:00
kern_cons.c Fix typos in VTY constant names from r268158 2014-07-02 14:47:48 +00:00
kern_context.c In order to maximize the re-usability of kernel code in user space this 2011-09-16 13:58:51 +00:00
kern_cpu.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_cpuset.c Permit changing cpu mask for cpu set 1 in presence of drivers 2014-06-22 11:32:23 +00:00
kern_ctf.c Remove the support for using non-mpsafe filesystem modules. 2012-10-22 17:50:54 +00:00
kern_descrip.c Don't zero fd_nfiles during fdp destruction. 2014-07-10 21:05:45 +00:00
kern_dtrace.c Commit the rest of the changes that were intended to be part of r266826. 2014-05-29 01:42:22 +00:00
kern_environment.c Make getenv_*() functions and respectively TUNABLE_*_FETCH() macros not 2013-11-01 10:32:33 +00:00
kern_et.c Fix build breakage. Apparently all ARM configs build kern_et.c, but only a 2014-04-02 17:34:17 +00:00
kern_event.c Hide internal details of sbintime_t implementation wrapping INT64_MAX into 2014-04-12 23:29:29 +00:00
kern_exec.c Don't call crdup nor uifind under vnode lock. 2014-07-07 14:03:30 +00:00
kern_exit.c Eliminate plim and vtmp local vars in exit1. 2014-07-10 22:54:38 +00:00
kern_fail.c Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs. 2011-11-07 06:44:47 +00:00
kern_ffclock.c Revise the sysctl handling code and restructure the hierarchy of sysctls 2011-12-01 07:19:13 +00:00
kern_fork.c Make fdunshare accept only td parameter. 2014-06-28 05:41:53 +00:00
kern_gzio.c Remove the support for using non-mpsafe filesystem modules. 2012-10-22 17:50:54 +00:00
kern_hhook.c Move hhook's per-vnet initialisation to an earlier SYSINIT SI_SUB stage to 2013-06-15 10:08:34 +00:00
kern_idle.c
kern_intr.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_jail.c Remove AppleTalk support. 2014-03-14 06:29:43 +00:00
kern_khelp.c Cleanup and simplification in khelp_{register|deregister}_helper(). No 2013-06-15 06:45:17 +00:00
kern_kthread.c Do not use potentially stale thread in kthread_add() 2013-08-17 17:02:43 +00:00
kern_ktr.c Drop KTR records when we're in the debugger so that the debugger isn't 2014-07-02 22:13:07 +00:00
kern_ktrace.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_linker.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_lock.c - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging 2013-11-25 07:38:45 +00:00
kern_lockf.c Correct the problem reported by test16 from 2014-07-08 08:10:15 +00:00
kern_lockstat.c - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging 2013-11-25 07:38:45 +00:00
kern_loginclass.c Initialize loginclass mutex using MTX_SYSINIT instead of using SI_SUB_CPU. 2014-05-14 09:03:02 +00:00
kern_malloc.c Compile fixes: 2014-06-28 17:36:18 +00:00
kern_mbuf.c Improve reference counting of EXT_SFBUF pages attached to mbufs. 2014-07-11 19:40:50 +00:00
kern_mib.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_module.c Fix a typo. 2012-08-22 20:01:57 +00:00
kern_mtxpool.c Garbage collect mtxpool_lockbuilder, the mutex pool historically used 2014-05-02 07:57:40 +00:00
kern_mutex.c - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging 2013-11-25 07:38:45 +00:00
kern_ntptime.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_osd.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_physio.c Fix some issues in change 254760 pointed out by Bruce Evans: 2013-08-29 16:41:40 +00:00
kern_pmc.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_poll.c Introduce a procedural interface to the ifnet structure. The new 2014-06-02 17:54:39 +00:00
kern_priv.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_proc.c Unconditionally initialize addr to handle the case of changed map 2014-07-10 11:20:24 +00:00
kern_prot.c Style fix 2012-11-14 10:33:12 +00:00
kern_racct.c dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE 2013-11-26 08:46:27 +00:00
kern_rangelock.c Change the queue of locks in kern_rangelock.c from holding lock requests in 2013-08-15 20:19:17 +00:00
kern_rctl.c Add CPU percentage limit enforcement to RCTL. The resouce name is "pcpu". 2012-10-26 16:01:08 +00:00
kern_resource.c rlimit: avoid unnecessary copying of rlimits 2013-12-13 20:54:45 +00:00
kern_rmlock.c - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging 2013-11-25 07:38:45 +00:00
kern_rwlock.c Drop the 3rd clause from all 3 clause BSD licenses where I am the sole 2014-02-05 18:13:27 +00:00
kern_sdt.c Print a backtrace if the SDT(9) stub gets called so that there's at least 2014-02-22 01:41:45 +00:00
kern_sema.c
kern_sharedpage.c Remove the deprecated VM_ALLOC_RETRY flag for the vm_page_grab(9). 2013-08-22 07:39:53 +00:00
kern_shutdown.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_sig.c Perform a lockless check in sigacts_shared. 2014-07-01 06:29:15 +00:00
kern_switch.c Add a comment on why inlining critical_enter() may not be a good idea 2012-12-09 04:54:22 +00:00
kern_sx.c - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging 2013-11-25 07:38:45 +00:00
kern_synch.c use saner calculations in should_yield 2013-11-26 14:00:50 +00:00
kern_syscalls.c
kern_sysctl.c Don't make a temporary copy of fixed sysctl strings. 2014-07-10 21:46:57 +00:00
kern_tc.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
kern_thr.c Stop treating td_sigmask specially for the purposes of new thread 2012-05-26 20:03:47 +00:00
kern_thread.c Remove ia64. 2014-07-07 00:27:09 +00:00
kern_time.c Fix VIRTUAL and PROF interval timers for short intervals, broken at r247903. 2014-04-16 18:37:46 +00:00
kern_timeout.c If we're doing RSS then ensure that the callwheel swi's are CPU pinned. 2014-06-30 04:25:51 +00:00
kern_umtx.c Fix comments. 2014-03-19 12:45:40 +00:00
kern_uuid.c Fix a bug in be_uuid_dec(); it called le16dec() instead of be16dec(), 2014-02-13 22:24:36 +00:00
kern_xxx.c
ksched.c sched_rr_interval() seems always returned period in hz ticks, but same 2012-08-10 18:19:57 +00:00
link_elf_obj.c Add a mmap flag (MAP_32BIT) on 64-bit platforms to request that a mapping use 2013-09-09 18:11:59 +00:00
link_elf.c Remove ia64. 2014-07-07 00:27:09 +00:00
linker_if.m
Make.tags.inc Remove AppleTalk support. 2014-03-14 06:29:43 +00:00
Makefile
makesyscalls.sh Fix syscalls that can be loaded as kernel modules - they were not given 2013-12-15 23:19:42 +00:00
md4c.c
md5c.c
p1003_1b.c In order to maximize the re-usability of kernel code in user space this 2011-09-16 13:58:51 +00:00
posix4_mib.c
sched_4bsd.c Given that as of r258002 the last external user is gone, make sched_lock 2014-04-29 20:51:57 +00:00
sched_ule.c Micro-manage clang to get the expected inlining for cpu_search(). 2014-07-03 11:06:27 +00:00
serdev_if.m
stack_protector.c
subr_acl_nfs4.c Fix bug where NFSv4 ACL enforcement code wouldn't unconditionally 2012-04-17 14:54:00 +00:00
subr_acl_posix1e.c Add module load/unload stubs. 2012-03-13 20:27:48 +00:00
subr_autoconf.c
subr_blist.c Remove reference to the rlist code from comments, and fix a typo visible 2013-02-05 20:08:33 +00:00
subr_bufring.c
subr_bus_dma.c Add bus_dmamap_load_ma() function to load map with the array of 2013-10-27 21:39:16 +00:00
subr_bus.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
subr_busdma_bufalloc.c Replace kernel virtual address space allocation with vmem. This provides 2013-08-07 06:21:20 +00:00
subr_capability.c Remove duplicated includes. 2014-06-26 13:57:44 +00:00
subr_clock.c Fix a 2038 bug. 2014-05-01 22:28:14 +00:00
subr_counter.c Create two public UMA_ZONE_PCPU zones: 64 bit sized and pointer sized. 2014-02-10 19:59:46 +00:00
subr_devstat.c change defaule permissions on /dev/devstat. while i'm here remove 2014-03-24 18:13:41 +00:00
subr_disk.c
subr_dummy_vdso_tc.c Implement mechanism to export some kernel timekeeping data to 2012-06-22 07:06:40 +00:00
subr_eventhandler.c
subr_fattime.c Fix a comment typo; conversion tables are for leap years, not leap seconds. 2014-04-20 13:37:22 +00:00
subr_firmware.c Correct sizeof usage 2012-06-25 05:41:16 +00:00
subr_hash.c Convert panic()s to KASSERT()s. This is an optimisation for 2012-01-23 16:31:46 +00:00
subr_hints.c Preserve one character space for a trailing '\0'. 2014-02-14 20:54:03 +00:00
subr_kdb.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
subr_kobj.c As it turns out, r186347 actually is insufficient to avoid the use of the 2011-11-15 20:11:03 +00:00
subr_lock.c Drop the 3rd clause from all 3 clause BSD licenses where I am the sole 2014-02-05 18:13:27 +00:00
subr_log.c MFcalloutng (r244255 by mav, with minor changes): 2013-03-04 16:07:55 +00:00
subr_mbpool.c All mbuf external free functions never fail, so let them be void. 2014-07-11 13:58:48 +00:00
subr_mchain.c Mechanically substitute flags from historic mbuf allocator with 2012-12-05 08:04:20 +00:00
subr_module.c
subr_msgbuf.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
subr_param.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
subr_pcpu.c Create two public UMA_ZONE_PCPU zones: 64 bit sized and pointer sized. 2014-02-10 19:59:46 +00:00
subr_pctrie.c - Add a new general purpose path-compressed radix trie which can be used 2013-05-12 04:05:01 +00:00
subr_power.c
subr_prf.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
subr_prof.c Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs. 2011-11-07 15:43:11 +00:00
subr_rman.c Declaration whitespace changes for style(9). 2014-07-07 22:02:39 +00:00
subr_rtc.c Core structure and functions to support a feed-forward clock within the kernel. 2011-11-19 14:10:16 +00:00
subr_sbuf.c Implement Short/Small String Optimization in SBUF(9) and change lengths and 2014-07-10 13:08:51 +00:00
subr_scanf.c Xen netback driver rewrite. 2012-01-26 16:35:09 +00:00
subr_sglist.c Add sglist_append_bio(9) to append a struct bio's data to a sglist 2014-01-13 04:41:08 +00:00
subr_sleepqueue.c sysctl subsystem uses sxlocks so avoid to setup dynamic sysctl nodes 2014-06-24 15:16:55 +00:00
subr_smp.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
subr_stack.c Constify stack argument for functions that don't modify it. 2011-11-16 19:06:55 +00:00
subr_syscall.c Update kernel inclusions of capability.h to use capsicum.h instead; some 2014-03-16 10:55:57 +00:00
subr_taskqueue.c Pin the right thread. 2014-06-01 04:11:05 +00:00
subr_terminal.c In order to get vt(4) a bit closer to the feature set provided by sc(4), 2014-06-27 19:57:57 +00:00
subr_trap.c Fix two issues with /dev/mem access on amd64, both causing kernel page 2014-03-21 14:25:09 +00:00
subr_turnstile.c dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE 2013-11-26 08:46:27 +00:00
subr_uio.c Remove zero-copy sockets code. It only worked for anonymous memory, 2013-09-16 06:25:54 +00:00
subr_unit.c Move the definition of the struct unrhdr into a separate header file, 2013-08-30 07:37:45 +00:00
subr_vmem.c Do not set M_BESTFIT if a strategy has already been provided. This 2014-04-16 21:39:43 +00:00
subr_witness.c Remove ia64. 2014-07-07 00:27:09 +00:00
sys_capability.c Update kernel inclusions of capability.h to use capsicum.h instead; some 2014-03-16 10:55:57 +00:00
sys_generic.c fd: replace fd_nfiles with fd_lastfile where appropriate 2014-06-22 01:31:55 +00:00
sys_pipe.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
sys_procdesc.c Implement kqueue(2) for procdesc(4). 2014-04-07 18:10:49 +00:00
sys_process.c Extend the support for exempting processes from being killed when swap is 2013-09-19 18:53:42 +00:00
sys_socket.c The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare 2013-10-26 17:58:36 +00:00
syscalls.c Regen per r263318. 2014-03-18 21:34:11 +00:00
syscalls.master Remove dead code from umtx support: 2014-03-18 21:32:03 +00:00
systrace_args.c Regen per r263318. 2014-03-18 21:34:11 +00:00
sysv_ipc.c
sysv_msg.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
sysv_sem.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
sysv_shm.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
tty_compat.c
tty_info.c Fix whitespace inconsistencies in TTY code. 2012-02-06 18:15:46 +00:00
tty_inq.c Use strchr() and strrchr(). 2012-01-02 12:12:10 +00:00
tty_outq.c
tty_pts.c Make sendfile() a method in the struct fileops. Currently only 2013-08-15 07:54:31 +00:00
tty_tty.c
tty_ttydisc.c Correct SIGTTIN handling. 2012-10-25 09:05:21 +00:00
tty.c Update kernel inclusions of capability.h to use capsicum.h instead; some 2014-03-16 10:55:57 +00:00
uipc_accf.c
uipc_debug.c Fix socket buffer timeouts precision using the new sbintime_t KPI instead 2013-09-01 23:34:53 +00:00
uipc_domain.c - Implement two new system calls: 2013-03-02 21:11:30 +00:00
uipc_mbuf2.c Mechanically substitute flags from historic mbuf allocator with 2012-12-05 08:04:20 +00:00
uipc_mbuf.c Improve reference counting of EXT_SFBUF pages attached to mbufs. 2014-07-11 19:40:50 +00:00
uipc_mqueue.c Update kernel inclusions of capability.h to use capsicum.h instead; some 2014-03-16 10:55:57 +00:00
uipc_sem.c Update kernel inclusions of capability.h to use capsicum.h instead; some 2014-03-16 10:55:57 +00:00
uipc_shm.c Update kernel inclusions of capability.h to use capsicum.h instead; some 2014-03-16 10:55:57 +00:00
uipc_sockbuf.c Don't leave the padding between the msg header and the cmsg data, 2014-07-08 21:54:23 +00:00
uipc_socket.c Simplify wait/nowait code, eventually killing last remnant of 2014-01-16 13:45:41 +00:00
uipc_syscalls.c Improve reference counting of EXT_SFBUF pages attached to mbufs. 2014-07-11 19:40:50 +00:00
uipc_usrreq.c Check lower bound of cmsg_len. 2014-06-27 05:04:36 +00:00
vfs_acl.c Update kernel inclusions of capability.h to use capsicum.h instead; some 2014-03-16 10:55:57 +00:00
vfs_aio.c Update kernel inclusions of capability.h to use capsicum.h instead; some 2014-03-16 10:55:57 +00:00
vfs_bio.c - Modify vm_page_unwire() and vm_page_enqueue() to directly accept 2014-06-16 18:15:27 +00:00
vfs_cache.c Fix accounting for the negative cache entries when reusing v_cache_dd. 2013-12-27 17:09:59 +00:00
vfs_cluster.c When allocating a pbuf for the cluster write, do not sleep waiting 2013-08-27 01:31:12 +00:00
vfs_default.c - Convert the bufobj lock to rwlock. 2013-05-31 00:43:41 +00:00
vfs_export.c Further refine the handling of stop signals in the NFS client. The 2013-02-21 19:02:50 +00:00
vfs_extattr.c Update kernel inclusions of capability.h to use capsicum.h instead; some 2014-03-16 10:55:57 +00:00
vfs_hash.c Add exported vfs_hash_index() function, which calculates the canonical 2013-01-14 05:41:40 +00:00
vfs_init.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
vfs_lookup.c Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
vfs_mount.c Use proper MFSNAMELEN for fs type. 2014-04-12 21:39:17 +00:00
vfs_mountroot.c Move root_mount_hold() functionality to separate mutex. 2014-06-11 08:14:08 +00:00
vfs_subr.c Implement simple direct-mapped cache for popular filesystem identifiers to 2014-06-12 12:43:48 +00:00
vfs_syscalls.c fd: replace fd_nfiles with fd_lastfile where appropriate 2014-06-22 01:31:55 +00:00
vfs_vnops.c In msdosfs_setattr(), add a check for result of the utimes(2) 2014-06-17 07:11:00 +00:00
vnode_if.src If filesystem declares that it supports shared locking for writes, use 2013-11-09 20:36:29 +00:00