Adrian Chadd 32766cd281 Also make kern.maxfilesperproc a boot time tunable. ...

Auto-tuning threshold discussions aside, it turns out that if you want
to lower this on say, rather memory-packed machines, you either set maxusers
or kern.maxfiles, or you set it in sysctl.  The former is a non-exact
way to tune this; the latter doesn't actually affect anything in the
startup scripts.

This first occured because I wondered why the hell screen would take upwards
of 10 seconds to spawn a new screen.  I then found python doing the same
thing during fork/exec of child processes - it calls close() on each FD
up to the current openfiles limit.  On a 1TB machine this is like, 26 million
FDs per process.  Ugh.

So:

* This allows it to be set early in /boot/loader.conf;
* It can be used to work around the ridiculous situation of
  screen, python, etc doing a close() on potentially millions of FDs
  even though you only have four open.

Tested:

* 4GB, 32GB, 64GB, 128GB, 384GB, 1TB systems with autotune, ensuring
  screen and python forking doesn't result in some pretty hilariously
  bad behaviour.

TODO:

* Note that the default login.conf sets openfiles-cur to unlimited,
  effectively obeying kern.maxfilesperproc.  Perhaps we should fix
  this.

* .. and even if we do, we need to also ensure that daemons get
  a soft limit of something reasonable and capped - they can request
  more FDs themselves.

MFC after:	1 week
Sponsored by:	Norse Corp, Inc.

2015-09-10 04:05:58 +00:00

..

bus_if.m

Add a bus method to fetch the VM domain for the given device/bus.

2014-10-09 05:33:25 +00:00

capabilities.conf

Add futimens and utimensat system calls.

2015-01-23 21:07:08 +00:00

clock_if.m

…

cpufreq_if.m

…

device_if.m

Change the default method for device_quiesce() to return 0 instead of

2015-01-08 21:46:28 +00:00

genassym.sh

genassym.sh: call nm(1) with NMFLAGS.

2015-08-14 22:57:13 +00:00

imgact_aout.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

imgact_binmisc.c

At the suggestion of jhb, replace atomic_set/clear calls with use of

2015-06-24 15:52:26 +00:00

imgact_elf32.c

…

imgact_elf64.c

…

imgact_elf.c

Follow-up to r287442: Move sysctl to compiled-once file

2015-09-07 16:44:28 +00:00

imgact_gzip.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

imgact_shell.c

Allow multiple image activators to run on the same execution by changing

2014-09-04 21:31:25 +00:00

inflate.c

…

init_main.c

Remove now obsolete comment.

2015-08-28 20:06:58 +00:00

init_sysent.c

Add an initial NUMA affinity/policy configuration for threads and processes.

2015-07-11 15:21:37 +00:00

kern_acct.c

…

kern_alq.c

Prevent alq from panic when the invalid alq_file path specified.

2014-04-05 16:54:47 +00:00

kern_clock.c

Initialize ticks so that it wraps 10 minutes after boot to increase the

2015-02-05 01:43:21 +00:00

kern_clocksource.c

Fix typo in comment.

2015-07-20 09:37:42 +00:00

kern_condvar.c

Revert r282971. It depends on condvar consumers not destroying condvars

2015-05-21 16:43:26 +00:00

kern_conf.c

Fix for out of order device destruction notifications when using the

2015-03-22 13:11:56 +00:00

kern_cons.c

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten

2015-05-22 17:05:21 +00:00

kern_context.c

…

kern_cpu.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_cpuset.c

Un-static cpuset_which() - it's useful in other contexts, such as some

2015-06-26 04:14:05 +00:00

kern_ctf.c

Don't specify a resid parameter if we're just going to ignore it. Instead,

2015-02-20 20:49:00 +00:00

kern_descrip.c

fd: make rights a mandatory argument to fgetvp_rights

2015-09-07 20:05:56 +00:00

kern_dtrace.c

Commit the rest of the changes that were intended to be part of r266826.

2014-05-29 01:42:22 +00:00

kern_dump.c

Factor out duplicated code from dumpsys() on each architecture into generic

2015-01-07 01:01:39 +00:00

kern_environment.c

Test if 'env' is NULL before doing memset() and strlen(),

2014-10-23 18:23:50 +00:00

kern_et.c

Trivial change / forced-commit to document prior change that slipped in

2015-03-16 19:29:19 +00:00

kern_event.c

Exit notification for EVFILT_PROC removes knote from the knlist. In

2015-09-01 14:05:29 +00:00

kern_exec.c

Follow-up to r287442: Move sysctl to compiled-once file

2015-09-07 16:44:28 +00:00

kern_exit.c

When the wait*(2) syscalls wait for any process (P_ALL), they should

2015-08-12 20:08:54 +00:00

kern_fail.c

Use a regular sbuf + SYSCTL_OUT() rather than sbuf_new_for_sysctl() with

2015-03-16 19:18:45 +00:00

kern_ffclock.c

The SYSCTL data pointers can come from userspace and must not be

2014-10-28 12:00:39 +00:00

kern_fork.c

Make kstack_pages a tunable on arm, x86, and powepc. On i386, the

2015-08-10 17:18:21 +00:00

kern_gzio.c

Move zlib.c from net to libkern.

2015-04-22 14:38:58 +00:00

kern_hhook.c

…

kern_idle.c

…

kern_intr.c

The part of r285680 which removed release semantic for two stores to

2015-07-21 14:39:34 +00:00

kern_jail.c

Add support to the jail framework to be able to mount linsysfs(5) and

2015-07-19 08:52:35 +00:00

kern_khelp.c

…

kern_kthread.c

Limit rights on process descriptors.

2015-07-31 10:21:58 +00:00

kern_ktr.c

Expand ktr_mask to be a 64-bit unsigned integer.

2015-05-22 11:09:41 +00:00

kern_ktrace.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_linker.c

Revert r285125 until rmlocks get fixed.

2015-07-30 19:52:43 +00:00

kern_lock.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_lockf.c

Improve style and fix a possible use-after-free case introduced in r268384

2015-01-10 06:48:35 +00:00

kern_lockstat.c

Consistently use a reader/writer flag for lockstat probes in rwlock(9) and

2015-07-19 22:24:33 +00:00

kern_loginclass.c

cred: add proc_set_cred helper

2015-03-16 00:10:03 +00:00

kern_malloc.c

The vmem callback to reclaim kmem arena address space on low or

2015-05-09 20:08:36 +00:00

kern_mbuf.c

Fix integer truncation bug in malloc(9)

2015-04-01 12:42:26 +00:00

kern_mib.c

Huge cleanup of random(4) code.

2015-06-30 17:00:45 +00:00

kern_module.c

…

kern_mtxpool.c

Garbage collect mtxpool_lockbuilder, the mutex pool historically used

2014-05-02 07:57:40 +00:00

kern_mutex.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_ntptime.c

Use the monotonic (uptime) counter rather than time-of-day to measure elapsed

2015-07-12 18:38:17 +00:00

kern_numa.c

Add an initial NUMA affinity/policy configuration for threads and processes.

2015-07-11 15:21:37 +00:00

kern_osd.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_physio.c

Rewrite physio() to not allocate pbufs for unmapped I/O.

2015-04-21 10:55:53 +00:00

kern_pmc.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_poll.c

When a kernel has DEVICE_POLLING turned on but no drivers have

2015-04-14 14:22:34 +00:00

kern_priv.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

kern_proc.c

The si_status field of the siginfo_t, provided by the waitid(2) and

2015-07-18 09:02:50 +00:00

kern_procctl.c

If process becomes reaper (procctl(PROC_REAP_ACQUIRE)) while already

2015-08-20 22:44:26 +00:00

kern_prot.c

Get rid of lim_update_thread and cred_update_thread.

2015-07-16 14:30:11 +00:00

kern_racct.c

nit: Rename racct_alloc_resource to racct_adjust_resource.

2015-06-14 08:33:14 +00:00

kern_rangelock.c

…

kern_rctl.c

Add kern.racct.enable tunable and RACCT_DISABLED config option.

2015-04-29 10:23:02 +00:00

kern_resource.c

Get rid of lim_update_thread and cred_update_thread.

2015-07-16 14:30:11 +00:00

kern_rmlock.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_rwlock.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_sdt.c

Print a backtrace if the SDT(9) stub gets called so that there's at least

2014-02-22 01:41:45 +00:00

kern_sema.c

…

kern_sharedpage.c

Copy the fencing of the algorithm to do lock-less update and reading

2015-08-04 12:33:51 +00:00

kern_shutdown.c

- Make 'struct buf *buf' private to vfs_bio.c. Having a global variable

2015-07-29 02:26:57 +00:00

kern_sig.c

Unignore signals when starting CloudABI processes.

2015-08-12 11:30:31 +00:00

kern_switch.c

Revert for r277213:

2015-01-22 11:12:42 +00:00

kern_sx.c

Don't modify curthread->td_locks unless INVARIANTS is enabled.

2015-08-02 00:03:08 +00:00

kern_synch.c

Remove several write-only variables, all reported by the gcc 4.9

2015-05-29 13:24:17 +00:00

kern_syscalls.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

kern_sysctl.c

Revert r285125 until rmlocks get fixed.

2015-07-30 19:52:43 +00:00

kern_tc.c

If a specific timecounter has been chosen via sysctl, and a new timecounter

2015-08-12 20:50:20 +00:00

kern_thr.c

Add an API for easily creating userspace threads in kernelspace.

2015-07-20 10:20:04 +00:00

kern_thread.c

Get rid of lim_update_thread and cred_update_thread.

2015-07-16 14:30:11 +00:00

kern_time.c

Fix an off by one in ppsratecheck(). If you asked for N=1 you'd get one,

2015-01-11 20:48:29 +00:00

kern_timeout.c

callout_reset: fix a reversed check for cc_exec_cancel

2015-09-01 09:27:14 +00:00

kern_umtx.c

Fix bad arithmetic in umtx_key_get() to compute object offset.

2015-08-04 06:01:13 +00:00

kern_uuid.c

Fix a bug in be_uuid_dec(); it called le16dec() instead of be16dec(),

2014-02-13 22:24:36 +00:00

kern_xxx.c

…

ksched.c

Use P1B_PRIO_MAX to designate max posix priority for the RR/FIFO

2015-08-30 18:02:57 +00:00

link_elf_obj.c

Move zlib.c from net to libkern.

2015-04-22 14:38:58 +00:00

link_elf.c

preload_search_info: make sure mod is set

2015-08-21 15:57:57 +00:00

linker_if.m

…

Make.tags.inc

Remove AppleTalk support.

2014-03-14 06:29:43 +00:00

Makefile

…

makesyscalls.sh

Import the CloudABI datatypes and create a system call table.

2015-07-09 07:20:15 +00:00

md4c.c

…

md5c.c

…

p1003_1b.c

In preparation for switching linuxulator to the use the native 1:1

2015-05-24 14:44:06 +00:00

posix4_mib.c

…

sched_4bsd.c

kgdb uses td_oncpu to determine if a thread is running and should use

2015-08-03 20:43:36 +00:00

sched_ule.c

Summary: Add the interactivity equations to the header comment for our

2015-08-26 16:36:41 +00:00

serdev_if.m

…

stack_protector.c

Use nitems() macro instead of __arraycount()

2015-06-16 20:19:00 +00:00

subr_acl_nfs4.c

Expose an interface to determine if an ACE is inherited.

2015-09-04 00:14:20 +00:00

subr_acl_posix1e.c

…

subr_autoconf.c

…

subr_blist.c

…

subr_bufring.c

…

subr_bus_dma.c

Add bus_dmamap_load_ma() function to load map with the array of

2013-10-27 21:39:16 +00:00

subr_bus.c

Huge cleanup of random(4) code.

2015-06-30 17:00:45 +00:00

subr_busdma_bufalloc.c

Fix integer truncation bug in malloc(9)

2015-04-01 12:42:26 +00:00

subr_capability.c

Remove duplicated includes.

2014-06-26 13:57:44 +00:00

subr_clock.c

For architectures where time_t is wide enough, in particular, 64bit

2014-12-12 09:37:18 +00:00

subr_counter.c

Create two public UMA_ZONE_PCPU zones: 64 bit sized and pointer sized.

2014-02-10 19:59:46 +00:00

subr_devstat.c

Fix multiple incorrect SYSCTL arguments in the kernel:

2014-10-21 07:31:21 +00:00

subr_disk.c

…

subr_dummy_vdso_tc.c

Update the vdso timehands only via tc_windup().

2015-01-20 03:54:30 +00:00

subr_eventhandler.c

…

subr_fattime.c

Where appropriate, use the modern terms for the one true time base

2014-12-21 05:07:11 +00:00

subr_firmware.c

Create a dedicated function for ensuring that cdir and rdir are populated.

2015-07-11 16:22:48 +00:00

subr_hash.c

…

subr_hints.c

Add a new device control utility for new-bus devices called devctl. This

2015-02-06 16:09:01 +00:00

subr_kdb.c

Fix multiple incorrect SYSCTL arguments in the kernel:

2014-10-21 07:31:21 +00:00

subr_kobj.c

…

subr_lock.c

Add _NEW flag to mtx(9), sx(9), rmlock(9) and rwlock(9).

2014-12-13 21:00:10 +00:00

subr_log.c

…

subr_mbpool.c

All mbuf external free functions never fail, so let them be void.

2014-07-11 13:58:48 +00:00

subr_mchain.c

…

subr_module.c

preload_search_info: make sure mod is set

2015-08-21 15:57:57 +00:00

subr_msgbuf.c

Pull in r267961 and r267973 again. Fix for issues reported will follow.

2014-06-28 03:56:17 +00:00

subr_param.c

Also make kern.maxfilesperproc a boot time tunable.

2015-09-10 04:05:58 +00:00

subr_pcpu.c

Create two public UMA_ZONE_PCPU zones: 64 bit sized and pointer sized.

2014-02-10 19:59:46 +00:00

subr_pctrie.c

…

subr_power.c

…

subr_prf.c

Add support for reading MAM attributes to camcontrol(8) and libcam(3).

2015-06-09 21:39:38 +00:00

subr_prof.c

The process spin lock currently has the following distinct uses:

2014-11-26 14:10:00 +00:00

subr_rman.c

Nuke the never-used RF_TIMESHARE feature, reducing the complexity of the

2014-07-16 22:18:19 +00:00

subr_rtc.c

…

subr_sbuf.c

The minimum sbuf buffer size is 2 bytes (a byte plus a nulterm), assert that.

2015-03-17 21:00:31 +00:00

subr_scanf.c

…

subr_sfbuf.c

Move KASSERT into locked region.

2014-08-11 15:06:07 +00:00

subr_sglist.c

Fix a couple of panics when detaching from a cxgbe/cxl interface that was

2015-01-26 16:26:28 +00:00

subr_sleepqueue.c

Revert for r277213:

2015-01-22 11:12:42 +00:00

subr_smp.c

The smp_rendezvous_cpus() function should ensure that all accesses

2015-07-21 22:56:46 +00:00

subr_stack.c

…

subr_syscall.c

The 'sa' argument to syscallret() is not unused.

2015-09-01 22:28:23 +00:00

subr_taskqueue.c

MFuser/delphij/zfs-arc-rebase@r281754:

2015-05-26 01:40:33 +00:00

subr_terminal.c

vt(4): Adjust the cursor position after changing the window size

2014-11-01 17:05:15 +00:00

subr_trap.c

racct: perform a lockless check for p_throttled

2015-07-13 22:52:11 +00:00

subr_turnstile.c

ddb: finish converting boolean values.

2015-05-21 15:16:18 +00:00

subr_uio.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

subr_unit.c

Move the definition of the struct unrhdr into a separate header file,

2013-08-30 07:37:45 +00:00

subr_vmem.c

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten

2015-05-22 17:05:21 +00:00

subr_witness.c

o Revert the other functional half of r239864, i. e. the merge of r134227

2015-07-24 15:13:21 +00:00

sys_capability.c

Introduce kern_cap_rights_limit().

2015-08-11 08:43:50 +00:00

sys_generic.c

Cover a race between doselwakeup() and selfdfree(). If doselwakeup()

2015-07-09 09:22:21 +00:00

sys_pipe.c

Make pipes in CloudABI work.

2015-07-29 17:18:27 +00:00

sys_procdesc.c

The si_status field of the siginfo_t, provided by the waitid(2) and

2015-07-18 09:02:50 +00:00

sys_process.c

Export current system call code and argument count for system call entry

2015-09-01 22:24:54 +00:00

sys_socket.c

In preparation of merging projects/sendfile, transform bare access to

2014-11-12 09:57:15 +00:00

syscalls.c

Regenerate syscalls.

2015-07-11 15:22:11 +00:00

syscalls.master

Regenerate syscalls.

2015-07-11 15:22:11 +00:00

systrace_args.c

Regenerate syscalls.

2015-07-11 15:22:11 +00:00

sysv_ipc.c

…

sysv_msg.c

Add kern.racct.enable tunable and RACCT_DISABLED config option.

2015-04-29 10:23:02 +00:00

sysv_sem.c

Add kern.racct.enable tunable and RACCT_DISABLED config option.

2015-04-29 10:23:02 +00:00

sysv_shm.c

sysvshm: fix up some whitespace issues and spurious initialisation

2015-07-02 19:14:30 +00:00

tty_compat.c

…

tty_info.c

…

tty_inq.c

…

tty_outq.c

…

tty_pts.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

tty_tty.c

tty: replace several curthread->td_proc with stored curproc

2015-07-06 18:53:56 +00:00

tty_ttydisc.c

…

tty.c

filedesc: simplify fget_unlocked & friends

2015-02-17 23:54:06 +00:00

uipc_accf.c

The accept filter code is not specific to the FreeBSD IPv4 network stack,

2014-07-26 19:27:34 +00:00

uipc_debug.c

Fix sb_state constant names as used e.g. to display in DDB ``show sockbuf''.

2015-07-21 09:57:13 +00:00

uipc_domain.c

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten

2015-05-22 17:05:21 +00:00

uipc_mbuf2.c

Add const-qualifiers for source mbuf argument in m_dup(), m_copym(),

2015-08-08 15:50:46 +00:00

uipc_mbuf.c

Add const-qualifiers for source mbuf argument in m_dup(), m_copym(),

2015-08-08 15:50:46 +00:00

uipc_mbufhash.c

Reduce header pollution.

2015-03-17 14:16:50 +00:00

uipc_mqueue.c

fd: remove filedesc argument from fdclose

2015-04-11 15:40:28 +00:00

uipc_sem.c

Call ksem_get() with initialized 'rights'.

2015-07-23 23:18:03 +00:00

uipc_shm.c

Add kern_shm_open().

2015-08-01 07:21:14 +00:00

uipc_sockbuf.c

Implement lockless resource limits.

2015-06-10 10:48:12 +00:00

uipc_socket.c

Make shutdown() return ENOTCONN as required by POSIX, part deux.

2015-07-27 13:17:57 +00:00

uipc_syscalls.c

Make shutdown() return ENOTCONN as required by POSIX, part deux.

2015-07-27 13:17:57 +00:00

uipc_usrreq.c

fd: make the common case in filecaps_copy work lockless

2015-09-07 20:02:56 +00:00

vfs_acl.c

Replace struct filedesc argument in getvnode with struct thread

2015-06-16 13:09:18 +00:00

vfs_aio.c

Mutex memory is not zeroed, add MTX_NEW.

2015-07-06 14:09:00 +00:00

vfs_bio.c

vfs: fill fallout from r286076

2015-07-30 15:43:26 +00:00

vfs_cache.c

Track changes to kern.maxvnodes and appropriately increase or decrease

2015-09-06 05:50:51 +00:00

vfs_cluster.c

Refactor unmapped buffer address handling.

2015-07-23 19:13:41 +00:00

vfs_default.c

vfs: use shared vnode locking when looking up ".." in vop_stdvptocnp

2015-07-04 15:46:39 +00:00

vfs_export.c

After the changes in r274118 make NOIP kernels compile by hiding an

2014-11-06 12:19:39 +00:00

vfs_extattr.c

Replace struct filedesc argument in getvnode with struct thread

2015-06-16 13:09:18 +00:00

vfs_hash.c

Track changes to kern.maxvnodes and appropriately increase or decrease

2015-09-06 05:50:51 +00:00

vfs_init.c

Revert r285125 until rmlocks get fixed.

2015-07-30 19:52:43 +00:00

vfs_lookup.c

Properly return ENOTDIR when calling *at() on a non-vnode.

2015-08-12 16:17:00 +00:00

vfs_mount.c

Make vfs_unmountall() unmount /dev after /, not before. The only

2015-08-24 13:18:13 +00:00

vfs_mountroot.c

An error of -1 from parse_mount() indicates that the specification

2015-08-27 04:25:27 +00:00

vfs_subr.c

Track changes to kern.maxvnodes and appropriately increase or decrease

2015-09-06 05:50:51 +00:00

vfs_syscalls.c

Decompose linkat()/renameat() rights to source and target.

2015-08-27 15:16:41 +00:00

vfs_vnops.c

For open("name", O_DIRECTORY | O_CREAT), do not try to create the

2015-09-09 19:31:08 +00:00

vnode_if.src

Catch up on r271387 and remove unused parameter from

2015-03-30 22:49:26 +00:00