0d3a835f3f
MAKEDEV: Add MAKEDEV glue for the ti(4) device nodes. ti.4: Update the ti(4) man page to include information on the TI_JUMBO_HDRSPLIT and TI_PRIVATE_JUMBOS kernel options, and also include information about the new character device interface and the associated ioctls. man9/Makefile: Add jumbo.9 and zero_copy.9 man pages and associated links. jumbo.9: New man page describing the jumbo buffer allocator interface and operation. zero_copy.9: New man page describing the general characteristics of the zero copy send and receive code, and what an application author should do to take advantage of the zero copy functionality. NOTES: Add entries for ZERO_COPY_SOCKETS, TI_PRIVATE_JUMBOS, TI_JUMBO_HDRSPLIT, MSIZE, and MCLSHIFT. conf/files: Add uipc_jumbo.c and uipc_cow.c. conf/options: Add the 5 options mentioned above. kern_subr.c: Receive side zero copy implementation. This takes "disposable" pages attached to an mbuf, gives them to a user process, and then recycles the user's page. This is only active when ZERO_COPY_SOCKETS is turned on and the kern.ipc.zero_copy.receive sysctl variable is set to 1. uipc_cow.c: Send side zero copy functions. Takes a page written by the user and maps it copy on write and assigns it kernel virtual address space. Removes copy on write mapping once the buffer has been freed by the network stack. uipc_jumbo.c: Jumbo disposable page allocator code. This allocates (optionally) disposable pages for network drivers that want to give the user the option of doing zero copy receive. uipc_socket.c: Add kern.ipc.zero_copy.{send,receive} sysctls that are enabled if ZERO_COPY_SOCKETS is turned on. Add zero copy send support to sosend() -- pages get mapped into the kernel instead of getting copied if they meet size and alignment restrictions. uipc_syscalls.c:Un-staticize some of the sf* functions so that they can be used elsewhere. (uipc_cow.c) if_media.c: In the SIOCGIFMEDIA ioctl in ifmedia_ioctl(), avoid calling malloc() with M_WAITOK. Return an error if the M_NOWAIT malloc fails. The ti(4) driver and the wi(4) driver, at least, call this with a mutex held. This causes witness warnings for 'ifconfig -a' with a wi(4) or ti(4) board in the system. (I've only verified for ti(4)). ip_output.c: Fragment large datagrams so that each segment contains a multiple of PAGE_SIZE amount of data plus headers. This allows the receiver to potentially do page flipping on receives. if_ti.c: Add zero copy receive support to the ti(4) driver. If TI_PRIVATE_JUMBOS is not defined, it now uses the jumbo(9) buffer allocator for jumbo receive buffers. Add a new character device interface for the ti(4) driver for the new debugging interface. This allows (a patched version of) gdb to talk to the Tigon board and debug the firmware. There are also a few additional debugging ioctls available through this interface. Add header splitting support to the ti(4) driver. Tweak some of the default interrupt coalescing parameters to more useful defaults. Add hooks for supporting transmit flow control, but leave it turned off with a comment describing why it is turned off. if_tireg.h: Change the firmware rev to 12.4.11, since we're really at 12.4.11 plus fixes from 12.4.13. Add defines needed for debugging. Remove the ti_stats structure, it is now defined in sys/tiio.h. ti_fw.h: 12.4.11 firmware. ti_fw2.h: 12.4.11 firmware, plus selected fixes from 12.4.13, and my header splitting patches. Revision 12.4.13 doesn't handle 10/100 negotiation properly. (This firmware is the same as what was in the tree previously, with the addition of header splitting support.) sys/jumbo.h: Jumbo buffer allocator interface. sys/mbuf.h: Add a new external mbuf type, EXT_DISPOSABLE, to indicate that the payload buffer can be thrown away / flipped to a userland process. socketvar.h: Add prototype for socow_setup. tiio.h: ioctl interface to the character portion of the ti(4) driver, plus associated structure/type definitions. uio.h: Change prototype for uiomoveco() so that we'll know whether the source page is disposable. ufs_readwrite.c:Update for new prototype of uiomoveco(). vm_fault.c: In vm_fault(), check to see whether we need to do a page based copy on write fault. vm_object.c: Add a new function, vm_object_allocate_wait(). This does the same thing that vm_object allocate does, except that it gives the caller the opportunity to specify whether it should wait on the uma_zalloc() of the object structre. This allows vm objects to be allocated while holding a mutex. (Without generating WITNESS warnings.) vm_object_allocate() is implemented as a call to vm_object_allocate_wait() with the malloc flag set to M_WAITOK. vm_object.h: Add prototype for vm_object_allocate_wait(). vm_page.c: Add page-based copy on write setup, clear and fault routines. vm_page.h: Add page based COW function prototypes and variable in the vm_page structure. Many thanks to Drew Gallatin, who wrote the zero copy send and receive code, and to all the other folks who have tested and reviewed this code over the years.
208 lines
7.6 KiB
C
208 lines
7.6 KiB
C
/*
|
|
* Copyright (c) 1991, 1993
|
|
* The Regents of the University of California. All rights reserved.
|
|
*
|
|
* This code is derived from software contributed to Berkeley by
|
|
* The Mach Operating System project at Carnegie-Mellon University.
|
|
*
|
|
* Redistribution and use in source and binary forms, with or without
|
|
* modification, are permitted provided that the following conditions
|
|
* are met:
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
* notice, this list of conditions and the following disclaimer.
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
* documentation and/or other materials provided with the distribution.
|
|
* 3. All advertising materials mentioning features or use of this software
|
|
* must display the following acknowledgement:
|
|
* This product includes software developed by the University of
|
|
* California, Berkeley and its contributors.
|
|
* 4. Neither the name of the University nor the names of its contributors
|
|
* may be used to endorse or promote products derived from this software
|
|
* without specific prior written permission.
|
|
*
|
|
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
|
|
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
|
|
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
* SUCH DAMAGE.
|
|
*
|
|
* from: @(#)vm_object.h 8.3 (Berkeley) 1/12/94
|
|
*
|
|
*
|
|
* Copyright (c) 1987, 1990 Carnegie-Mellon University.
|
|
* All rights reserved.
|
|
*
|
|
* Authors: Avadis Tevanian, Jr., Michael Wayne Young
|
|
*
|
|
* Permission to use, copy, modify and distribute this software and
|
|
* its documentation is hereby granted, provided that both the copyright
|
|
* notice and this permission notice appear in all copies of the
|
|
* software, derivative works or modified versions, and any portions
|
|
* thereof, and that both notices appear in supporting documentation.
|
|
*
|
|
* CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS"
|
|
* CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND
|
|
* FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE.
|
|
*
|
|
* Carnegie Mellon requests users of this software to return to
|
|
*
|
|
* Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU
|
|
* School of Computer Science
|
|
* Carnegie Mellon University
|
|
* Pittsburgh PA 15213-3890
|
|
*
|
|
* any improvements or extensions that they make and grant Carnegie the
|
|
* rights to redistribute these changes.
|
|
*
|
|
* $FreeBSD$
|
|
*/
|
|
|
|
/*
|
|
* Virtual memory object module definitions.
|
|
*/
|
|
|
|
#ifndef _VM_OBJECT_
|
|
#define _VM_OBJECT_
|
|
|
|
#include <sys/queue.h>
|
|
|
|
enum obj_type { OBJT_DEFAULT, OBJT_SWAP, OBJT_VNODE, OBJT_DEVICE, OBJT_PHYS,
|
|
OBJT_DEAD };
|
|
typedef u_char objtype_t;
|
|
|
|
/*
|
|
* Types defined:
|
|
*
|
|
* vm_object_t Virtual memory object.
|
|
*
|
|
* List of locks
|
|
* (c) const until freed
|
|
*
|
|
*/
|
|
|
|
struct vm_object {
|
|
TAILQ_ENTRY(vm_object) object_list; /* list of all objects */
|
|
TAILQ_HEAD(, vm_object) shadow_head; /* objects that this is a shadow for */
|
|
TAILQ_ENTRY(vm_object) shadow_list; /* chain of shadow objects */
|
|
TAILQ_HEAD(, vm_page) memq; /* list of resident pages */
|
|
int generation; /* generation ID */
|
|
vm_pindex_t size; /* Object size */
|
|
int ref_count; /* How many refs?? */
|
|
int shadow_count; /* how many objects that this is a shadow for */
|
|
int hash_rand; /* (c) hash table randomizer */
|
|
objtype_t type; /* type of pager */
|
|
u_short flags; /* see below */
|
|
u_short pg_color; /* (c) color of first page in obj */
|
|
u_short paging_in_progress; /* Paging (in or out) so don't collapse or destroy */
|
|
int resident_page_count; /* number of resident pages */
|
|
struct vm_object *backing_object; /* object that I'm a shadow of */
|
|
vm_ooffset_t backing_object_offset;/* Offset in backing object */
|
|
TAILQ_ENTRY(vm_object) pager_object_list; /* list of all objects of this pager type */
|
|
void *handle;
|
|
union {
|
|
/*
|
|
* VNode pager
|
|
*
|
|
* vnp_size - current size of file
|
|
*/
|
|
struct {
|
|
off_t vnp_size;
|
|
} vnp;
|
|
|
|
/*
|
|
* Device pager
|
|
*
|
|
* devp_pglist - list of allocated pages
|
|
*/
|
|
struct {
|
|
TAILQ_HEAD(, vm_page) devp_pglist;
|
|
} devp;
|
|
|
|
/*
|
|
* Swap pager
|
|
*
|
|
* swp_bcount - number of swap 'swblock' metablocks, each
|
|
* contains up to 16 swapblk assignments.
|
|
* see vm/swap_pager.h
|
|
*/
|
|
struct {
|
|
int swp_bcount;
|
|
} swp;
|
|
} un_pager;
|
|
};
|
|
|
|
/*
|
|
* Flags
|
|
*/
|
|
#define OBJ_ACTIVE 0x0004 /* active objects */
|
|
#define OBJ_DEAD 0x0008 /* dead objects (during rundown) */
|
|
#define OBJ_NOSPLIT 0x0010 /* dont split this object */
|
|
#define OBJ_PIPWNT 0x0040 /* paging in progress wanted */
|
|
#define OBJ_WRITEABLE 0x0080 /* object has been made writable */
|
|
#define OBJ_MIGHTBEDIRTY 0x0100 /* object might be dirty */
|
|
#define OBJ_CLEANING 0x0200
|
|
#define OBJ_OPT 0x1000 /* I/O optimization */
|
|
#define OBJ_ONEMAPPING 0x2000 /* One USE (a single, non-forked) mapping flag */
|
|
|
|
#define IDX_TO_OFF(idx) (((vm_ooffset_t)(idx)) << PAGE_SHIFT)
|
|
#define OFF_TO_IDX(off) ((vm_pindex_t)(((vm_ooffset_t)(off)) >> PAGE_SHIFT))
|
|
|
|
#ifdef _KERNEL
|
|
|
|
#define OBJPC_SYNC 0x1 /* sync I/O */
|
|
#define OBJPC_INVAL 0x2 /* invalidate */
|
|
#define OBJPC_NOSYNC 0x4 /* skip if PG_NOSYNC */
|
|
|
|
TAILQ_HEAD(object_q, vm_object);
|
|
|
|
extern struct object_q vm_object_list; /* list of allocated objects */
|
|
extern struct mtx vm_object_list_mtx; /* lock for object list and count */
|
|
|
|
extern vm_object_t kernel_object; /* the single kernel object */
|
|
extern vm_object_t kmem_object;
|
|
|
|
#endif /* _KERNEL */
|
|
|
|
#ifdef _KERNEL
|
|
void vm_freeze_copyopts(vm_object_t, vm_pindex_t, vm_pindex_t);
|
|
|
|
void vm_object_set_flag(vm_object_t object, u_short bits);
|
|
void vm_object_clear_flag(vm_object_t object, u_short bits);
|
|
void vm_object_pip_add(vm_object_t object, short i);
|
|
void vm_object_pip_subtract(vm_object_t object, short i);
|
|
void vm_object_pip_wakeup(vm_object_t object);
|
|
void vm_object_pip_wakeupn(vm_object_t object, short i);
|
|
void vm_object_pip_sleep(vm_object_t object, char *waitid);
|
|
void vm_object_pip_wait(vm_object_t object, char *waitid);
|
|
|
|
vm_object_t vm_object_allocate (objtype_t, vm_pindex_t);
|
|
vm_object_t vm_object_allocate_wait (objtype_t, vm_pindex_t, int);
|
|
void _vm_object_allocate (objtype_t, vm_pindex_t, vm_object_t);
|
|
boolean_t vm_object_coalesce (vm_object_t, vm_pindex_t, vm_size_t, vm_size_t);
|
|
void vm_object_collapse (vm_object_t);
|
|
void vm_object_deallocate (vm_object_t);
|
|
void vm_object_terminate (vm_object_t);
|
|
void vm_object_vndeallocate (vm_object_t);
|
|
void vm_object_set_writeable_dirty (vm_object_t);
|
|
void vm_object_init (void);
|
|
void vm_object_page_clean (vm_object_t, vm_pindex_t, vm_pindex_t, boolean_t);
|
|
void vm_object_page_remove (vm_object_t, vm_pindex_t, vm_pindex_t, boolean_t);
|
|
void vm_object_pmap_copy (vm_object_t, vm_pindex_t, vm_pindex_t);
|
|
void vm_object_pmap_copy_1 (vm_object_t, vm_pindex_t, vm_pindex_t);
|
|
void vm_object_pmap_remove (vm_object_t, vm_pindex_t, vm_pindex_t);
|
|
void vm_object_reference (vm_object_t);
|
|
void vm_object_shadow (vm_object_t *, vm_ooffset_t *, vm_size_t);
|
|
void vm_object_split(vm_map_entry_t);
|
|
void vm_object_madvise (vm_object_t, vm_pindex_t, int, int);
|
|
void vm_object_init2 (void);
|
|
#endif /* _KERNEL */
|
|
|
|
#endif /* _VM_OBJECT_ */
|