d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
42a69dcade
freebsd-skq/sys/kern
History
andre 42a69dcade Rewrite kern_sendfile() to work in two loops, the inner which turns as many 
VM pages into mbufs as it can -- up to the free send socket buffer space.
The outer loop then drops the whole mbuf chain into the send socket buffer,
calls tcp_output() on it and then waits until 50% of the socket buffer are
free again to repeat the cycle. This way tcp_output() gets the full amount
of data to work with and can issue up to 64K sends for TSO to chop up in
the network adapter without using any CPU cycles. Thus it gets very efficient
especially with the readahead the VM and I/O system do.

The previous sendfile(2) code simply looped over the file, turned each 4K
page into an mbuf and sent it off. This had the effect that TSO could only
generate 2 packets per send instead of up to 44 at its maximum of 64K.

Add experimental SF_MNOWAIT flag to sendfile(2) to return ENOMEM instead of
sleeping on mbuf allocation failures.

Benchmarking shows significant improvements (95% confidence):
 45% less cpu (or 1.81 times better) with new sendfile vs. old sendfile (non-TSO)
 83% less cpu (or 5.7 times better) with new sendfile vs. old sendfile (TSO)

(Sender AMD Opteron 852 (2.6GHz) with em(4) PCI-X-133 interface and receiver
DELL Poweredge SC1425 P-IV Xeon 3.2GHz with em(4) LOM connected back to back
at 1000Base-TX full duplex.)

Sponsored by:	TCP/IP Optimization Fundraise 2005
MFC after:	3 month
2006-11-02 16:53:26 +00:00
..
bus_if.m - Revert making bus_generic_add_child() the default for BUS_ADD_CHILD(). 2006-09-11 22:20:37 +00:00
clock_if.m
cpufreq_if.m
device_if.m
genassym.sh
imgact_aout.c Correct two vm object reference leaks in error cases. 2006-03-16 08:51:59 +00:00
imgact_elf32.c
imgact_elf64.c
imgact_elf.c Avoid a vm object reference leak in a rarely used code path. 2006-01-21 20:11:49 +00:00
imgact_gzip.c Maintain the lock on the vnode for most of exec_elfN_imgact(). 2005-12-24 04:57:50 +00:00
imgact_shell.c Fix a panic which could occur parsing #!-lines in a shell-script. If the 2005-06-19 02:21:03 +00:00
inflate.c Normalize a significant number of kernel malloc type names: 2005-10-31 15:41:29 +00:00
init_main.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
init_sysent.c Regenerate. 2006-10-17 02:28:58 +00:00
kern_acct.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_acl.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_alq.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_clock.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_condvar.c Fix a sleep queue race for KSE thread. 2006-02-23 00:13:58 +00:00
kern_conf.c Fix the race between devfs_fp_check and devfs_reclaim. Derefence the 2006-10-20 07:59:50 +00:00
kern_context.c
kern_cpu.c - Print message about cpufreq and timecounter TSC 2006-03-03 02:06:04 +00:00
kern_descrip.c return EBADF instead of successfully attaching (and then panicing) when 2006-09-24 02:29:53 +00:00
kern_environment.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_event.c remove unnecessary NULL check... 2006-09-25 01:29:48 +00:00
kern_exec.c The page queues lock is no longer required by vm_page_busy() or 2006-10-22 21:18:48 +00:00
kern_exit.c Move sigqueue_take() call into proc_reparent(), this fixed bugs where 2006-10-25 06:18:04 +00:00
kern_fork.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_idle.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_intr.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_jail.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_kse.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_kthread.c Don't do a PHOLD() in kthread_create() w/o a matching PRELE() in 2006-02-22 17:21:45 +00:00
kern_ktr.c Remove slightly oddly placed suser() call from the KTR/ALQ setup sysctl: 2006-09-09 16:09:01 +00:00
kern_ktrace.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_linker.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_lock.c If the buffer lock has waiters after the buffer has changed identity then 2006-10-02 02:06:27 +00:00
kern_lockf.c Print name of device instead of useless major/minor numbers. 2005-03-29 08:13:01 +00:00
kern_mac.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_malloc.c Increase usefulness of "show malloc" by moving from displaying the basic 2006-10-26 10:17:13 +00:00
kern_mbuf.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_mib.c mp_ncpus is always (properly) initialized, even on UP kernels, so just use it. 2005-08-21 18:03:31 +00:00
kern_module.c Address a problem I missed in removing Giant from the kernel linker. Not 2006-06-26 18:34:45 +00:00
kern_mtxpool.c
kern_mutex.c - When spinning on a spin lock, if the debugger is active or we are in a 2006-08-15 18:26:12 +00:00
kern_ntptime.c Explicitly acquire Giant around the ntp_gettime() and assert it in the 2005-05-28 14:34:41 +00:00
kern_physio.c
kern_pmc.c Fix -Wundef. 2005-12-04 02:12:43 +00:00
kern_poll.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_proc.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_prot.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_resource.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_rwlock.c Adjust td_locks for non-spin mutexes, rwlocks, and sx locks so that it is 2006-07-27 21:45:55 +00:00
kern_sema.c
kern_shutdown.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_sig.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_subr.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_switch.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_sx.c Add a new 'show sleepchain' ddb command similar to 'show lockchain' except 2006-08-15 18:29:01 +00:00
kern_synch.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_syscalls.c Make system call modules a bit more robust: 2006-08-01 16:32:20 +00:00
kern_sysctl.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_tc.c Commit the results of the typo hunt by Darren Pilgrim. 2006-08-04 07:56:35 +00:00
kern_thr.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_thread.c Remove member p_procscopegrp which is no longer used by libthr. 2006-10-27 05:45:44 +00:00
kern_time.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
kern_timeout.c Improve ktr(4) logging for callout(9) subsystem. Log all inserts and 2006-10-11 14:57:03 +00:00
kern_umtx.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
kern_uuid.c Separate functions with a newline. 2006-07-17 21:00:42 +00:00
kern_xxx.c
ksched.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
link_elf_obj.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
link_elf.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
linker_if.m
Make.tags.inc
Makefile Add support for the generated file systrace_args.c. 2006-08-05 19:25:14 +00:00
makesyscalls.sh Add a new set of macros <prefix>_AUE_<syscallname> to sysproto.h that 2006-08-15 17:09:32 +00:00
md4c.c
md5c.c Fix a panic on sparc64 related to inproper aligment - we cannot assume, 2006-03-30 18:45:50 +00:00
p1003_1b.c Backout the feature which can change thread's scheduling option, I really 2006-07-13 06:41:26 +00:00
posix4_mib.c
sched_4bsd.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
sched_core.c Add user priority loaning code to support priority propagation for 2006-08-25 06:12:53 +00:00
sched_ule.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
serdev_if.m MFp4: Add the ipend() method to the serdev I/F to allow umbrella 2006-04-23 22:12:39 +00:00
subr_acl_posix1e.c Update and reformat comments for POSIX.1e ACL utility routines. 2006-07-23 19:35:10 +00:00
subr_autoconf.c Add a mutex to protect the list of interrupt config hooks. We do assume 2006-07-19 18:53:56 +00:00
subr_blist.c
subr_bus.c Add a default method for BUS_ADD_CHILD() that just calls 2006-09-11 19:41:31 +00:00
subr_clist.c
subr_clock.c Use utc_offset() where applicable, and hide the internals of it 2006-10-02 18:23:37 +00:00
subr_devstat.c - Remove two mtx_asserts that can incorrectly trigger if 2005-05-03 10:58:05 +00:00
subr_disk.c Add a new I/O request - BIO_FLUSH, which basically tells providers below to 2006-10-31 21:11:21 +00:00
subr_eventhandler.c
subr_fattime.c Better naming of fattime conversion functions, they do convert to timespec 2006-10-24 10:27:23 +00:00
subr_firmware.c If linker_release_module() fails then we still hold a reference on 2006-06-25 12:36:21 +00:00
subr_hints.c Use a sleep mutex instead of an sx lock for the kernel environment. This 2006-07-09 21:42:58 +00:00
subr_kdb.c Add a funny sysctl: debug.kdb.trap_code . 2006-06-18 12:27:59 +00:00
subr_kobj.c Increment kobj_lookup_misses on a miss rather than decrementing it. 2005-12-29 18:00:42 +00:00
subr_lock.c Add a basic reader/writer lock implementation to the kernel. This 2006-01-27 23:13:26 +00:00
subr_log.c
subr_mbpool.c
subr_mchain.c Change API of mb_copy_t in libmchain so that netsmb can handle 2005-07-29 13:22:37 +00:00
subr_module.c
subr_msgbuf.c
subr_param.c Partially revert revision 1.66, which contained a change that did not 2005-10-14 19:15:10 +00:00
subr_pcpu.c Fix 'show allpcpu' ddb command on non-x86. CPU IDs are in the range 0 .. 2005-11-03 21:06:29 +00:00
subr_power.c General consensus is that it would be even better to run this in a 2005-11-09 16:22:56 +00:00
subr_prf.c Add a cnputs() function to write a string to the console with 2006-11-01 04:54:51 +00:00
subr_prof.c Change the addupc_*() functions to use the uintfptr_t type for pc rather 2005-12-16 22:08:32 +00:00
subr_rman.c - Fix rman_manage_region() to be a lot more intelligent. It now checks 2006-09-11 19:31:52 +00:00
subr_rtc.c Use utc_offset() where applicable, and hide the internals of it 2006-10-02 18:23:37 +00:00
subr_sbuf.c Make sbuf_copyin() return the number of bytes copied on success. 2005-12-23 11:49:53 +00:00
subr_scanf.c
subr_sleepqueue.c Print td_name instead of p_comm if td_name is non-empty for 2006-04-21 20:40:43 +00:00
subr_smp.c Rename the KDB_STOP_NMI kernel option to STOP_NMI and make it apply to all 2005-10-24 21:04:19 +00:00
subr_stack.c Correct typos 2006-05-28 22:15:28 +00:00
subr_taskqueue.c When starting up threads in taskqueue_start_threads create them 2006-05-24 22:11:07 +00:00
subr_trap.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
subr_turnstile.c Add a new 'show sleepchain' ddb command similar to 'show lockchain' except 2006-08-15 18:29:01 +00:00
subr_unit.c
subr_witness.c Introduce a spinlock for synchronizing access to the video output hardware 2006-09-13 15:48:15 +00:00
sys_generic.c Prevent IOC_IN with zero size argument (this is only supported 2006-10-14 19:01:55 +00:00
sys_pipe.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
sys_process.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
sys_socket.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
syscalls.c Regenerate. 2006-10-17 02:28:58 +00:00
syscalls.master o Add keyword volatile for user mutex owner field. 2006-10-17 02:24:47 +00:00
systrace_args.c Regenerate. 2006-10-17 02:28:58 +00:00
sysv_ipc.c
sysv_msg.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
sysv_sem.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
sysv_shm.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
tty_compat.c Move the old BSD4.3 tty compatibility from (!BURN_BRIDGES && COMPAT_43) 2006-01-10 09:19:10 +00:00
tty_conf.c
tty_cons.c Add a cnputs() function to write a string to the console with 2006-11-01 04:54:51 +00:00
tty_pts.c Increment nb_allocated while holding the pt_mtx lock to avoid races. 2006-11-01 16:50:13 +00:00
tty_pty.c Back out part of rev. 1.149. While adding a workaround in ptcopen() to 2006-10-04 05:43:39 +00:00
tty_subr.c
tty_tty.c Use ctty instead of just returning. ctty just has a simple open that 2006-09-27 16:41:15 +00:00
tty.c Make KSE a kernel option, turned on by default in all GENERIC 2006-10-26 21:42:22 +00:00
uipc_accf.c o setsockopt(2) cannot remove accept filter. [1] 2005-06-11 11:59:48 +00:00
uipc_cow.c Previously, nothing prevented the page that was returned by pmap_extract() 2005-10-23 07:41:56 +00:00
uipc_domain.c soreceive_generic(), and sopoll_generic(). Add new functions sosend(), 2006-07-24 15:20:08 +00:00
uipc_mbuf2.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
uipc_mbuf.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
uipc_mqueue.c Use mount interlock to protect all changes to mnt_flag and mnt_kern_flag. 2006-09-26 04:12:49 +00:00
uipc_sem.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
uipc_sockbuf.c Use sysctl_handle_long() instead of duplicating it's logic for 2006-09-06 21:59:36 +00:00
uipc_socket2.c Change two XXX's to two notes: the fact that SOCK_LOCK(so) == 2006-08-02 16:23:52 +00:00
uipc_socket.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
uipc_syscalls.c Rewrite kern_sendfile() to work in two loops, the inner which turns as many 2006-11-02 16:53:26 +00:00
uipc_usrreq.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
vfs_acl.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
vfs_aio.c MFP4 (with some minor changes): 2006-10-15 14:22:14 +00:00
vfs_bio.c Refactor vfs_setdirty(), creating vfs_setdirty_locked_object(). 2006-10-29 00:04:39 +00:00
vfs_cache.c Axe Giant from vn_fullpath(9). The vnode -> pathname lookup should be 2006-06-16 05:09:28 +00:00
vfs_cluster.c Replace PG_BUSY with VPO_BUSY. In other words, changes to the page's 2006-10-22 04:28:14 +00:00
vfs_default.c Don't try to obtain a reference to a nonexisting (NULL) mount structure in 2006-09-20 00:27:02 +00:00
vfs_export.c Use mount interlock to protect all changes to mnt_flag and mnt_kern_flag. 2006-09-26 04:12:49 +00:00
vfs_extattr.c The attempt to rename "." with MAC framework compiled in would cause attempt 2006-10-26 13:20:28 +00:00
vfs_hash.c In vfs_hash_get(): mount point should never be changed 2006-04-18 08:05:08 +00:00
vfs_init.c Remove duplicate security checks already performed in kern_kldload(). 2006-06-26 18:33:32 +00:00
vfs_lookup.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
vfs_mount.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
vfs_subr.c Add gjournal specific code to the UFS file system: 2006-10-31 21:48:54 +00:00
vfs_syscalls.c The attempt to rename "." with MAC framework compiled in would cause attempt 2006-10-26 13:20:28 +00:00
vfs_vnops.c Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h 2006-10-22 11:52:19 +00:00
vnode_if.src Remove two locking assertion entries that: 2006-05-31 14:06:06 +00:00