Commit Graph

12875 Commits

Author SHA1 Message Date
Jim Harris
84e7a2ebb7 Pad and align the callout_cpu mtx to its own cacheline to reduce false
sharing especially on the default CPU 0 callout_cpu structure.

This will be followed up by attilio@ with a conversion to the new struct
mtx_padalign but doing this manual conversion first gives an easy MFC
candidate since mtx_padalign is a more extensive system change.

Sponsored by:	Intel
Reviewed by:	jeff, attilio
MFC after:	1 week
2012-10-31 17:12:12 +00:00
Attilio Rao
7f44c61839 Give mtx(9) the ability to crunch different type of structures, with the
only constraint that they have a lock cookie named mtx_lock.
This name, then, becames reserved from the struct that wants to use the
mtx(9) KPI and other locking primitives cannot reuse it for their
members.

Namely such structs are the current struct mtx and the new
struct mtx_padalign.  The new structure will define an object which is
the same as the same layout of a struct mtx but will be allocated in
areas aligned to the cache line size and will be as big as a cache line.

This is supposed to give higher performance for highly contented mutexes
both spin or sleep (because of the adaptive spinning), where the cache
line contention results in too much traffic on the system bus.

The struct mtx_padalign can be used in a completely transparent way
with the mtx(9) KPI.

At the moment, a possibility to MFC the patch should be carefully
evaluated because this patch breaks the low level KPI
(not its representation though).

Discussed with:	jhb
Reviewed by:	jeff, andre
Reviewed by:	mdf (earlier version)
Tested by:	jimharris
2012-10-31 13:38:56 +00:00
Attilio Rao
5584e91718 Fixup r240246: hwpmc needs to retain the pinning until ASTs are not
executed. This means past the point where userret() is generally
executed.

Skip the td_pinned check if a callchain tracing is currently happening
and add a more robust check to pmc_capture_user_callchain() in order to
catch td_pinned leak past ast() in hwpmc case.

Reported and tested by:	fabient
MFC after:	1 week
X-MFC:	r240246
2012-10-30 15:10:50 +00:00
Attilio Rao
a049aa05c9 tdq_lock_pair() already does spinlock_enter() so migration is not
possible in sched_balance_pair(). Remove redundant sched_pin().

Reviewed by:	marius, jeff
2012-10-30 12:25:52 +00:00
Andre Oppermann
e8ad36aba4 In soreceive_stream() don't drop an already dequeued mbuf chain by
overwriting the return mbuf pointer with newly received data after
a loop.  Instead append the new mbuf chain to the existing one.

Fix up sb_lastrecord when dequeuing mbuf's so that sbappend_stream()
doesn't get confused.

For the remainder copy case in the mbuf delivery part deduct the
copied length len instead of the whole mbuf length.  Additionally
don't depend on 'n' being being available which isn't true in the
case of MSG_PEEK.

Fix the MSG_WAITALL case by comparing against sb_hiwat.  Before
it was looping for every receive as sb_lowat normally is zero.
Add comment about issue with (MSG_WAITALL | MSG_PEEK) which isn't
properly handled.

Submitted by:	trociny (except for the change in last paragraph)
2012-10-29 12:31:12 +00:00
Andre Oppermann
fdd1b7f52a Add logging for socket attach failures in sonewconn() during accept(2).
Include the pointer to the PCB so it can be attributed to a particular
application by corresponding it to "netstat -A" output.

MFC after:	2 weeks
2012-10-29 12:14:57 +00:00
Kevin Lo
a2c36a0234 Since the macro dtom() has been removed, fix comments about the dtom.
Reviewed by:	glebius
2012-10-29 10:04:28 +00:00
Andre Oppermann
14d7c5b11c Improve m_cat() by being able to also merge contents from M_EXT
mbuf's by doing proper testing with M_WRITABLE().

In m_collapse() replace an incomplete manual check for M_RDONLY
with the M_WRITABLE() macro that also tests for shared buffers
and other cases that make a particular mbuf immutable.

MFC after:	2 weeks
2012-10-28 18:38:51 +00:00
Davide Italiano
ba4be2110a The fields of struct timespec32 should be int32_t and not uint32_t.
Make this change.

Reviewed by:	bde, davidxu
Tested by:	pho
MFC after:	1 week
2012-10-27 23:42:41 +00:00
Edward Tomasz Napierala
36af98697d Add CPU percentage limit enforcement to RCTL. The resouce name is "pcpu".
It was implemented by Rudolf Tomori during Google Summer of Code 2012.
2012-10-26 16:01:08 +00:00
Ed Schouten
1da7bb41ed Correct SIGTTIN handling.
In the old TTY layer, SIGTTIN was correctly handled like this:

	while (data should be read) {
		send SIGTTIN if not foreground process group
		read data
	}

In the new TTY layer, however, this behaviour was changed, based on a
false interpretation of the standard:

	send SIGTTIN if not foreground process group
	while (data should be read) {
		read data
	}

Correct this by pushing tty_wait_background() into the ttydisc_read_*()
functions.

Reported by:	koitsu
PR:		kern/173010
MFC after:	2 weeks
2012-10-25 09:05:21 +00:00
Alfred Perlstein
7b6d92c0a0 Allow autotune maxusers > 384 on 64 bit machines
A default install on large memory machines with multiple 10gigE interfaces
were not being given enough mbufs to do full bandwidth TCP or NFS traffic.

To keep the value somewhat reasonable, we scale back the number of
maxuers by 1/6 past the 384 point.  This gives us enough mbufs for most
of our pretty basic 10gigE line-speed tests to complete.
2012-10-25 01:46:20 +00:00
Jim Harris
39f819e2fc Pad tdq_lock to avoid false sharing with tdq_load and tdq_cpu_idle.
This enables CPU searches (which read tdq_load) to operate independently
of any contention on the spinlock.  Some scheduler-intensive workloads
running on an 8C single-socket SNB Xeon show considerable improvement with
this change (2-3% perf improvement, 5-6% decrease in CPU util).

Sponsored by:	Intel
Reviewed by:	jeff
2012-10-24 18:36:41 +00:00
Andre Oppermann
e37e60c379 Replace the ill-named ZERO_COPY_SOCKET kernel option with two
more appropriate named kernel options for the very distinct
send and receive path.

"options SOCKET_SEND_COW" enables VM page copy-on-write based
sending of data on an outbound socket.

NB: The COW based send mechanism is not safe and may result
in kernel crashes.

"options SOCKET_RECV_PFLIP" enables VM kernel/userspace page
flipping for special disposable pages attached as external
storage to mbufs.

Only the naming of the kernel options is changed and their
corresponding #ifdef sections are adjusted.  No functionality
is added or removed.

Discussed with:	alc (mechanism and limitations of send side COW)
2012-10-23 14:19:44 +00:00
Ed Schouten
d7259a57bd Remove unused `vfslocked' variable.
I have no idea what this `vfslocked' thing means. I wonder how it ended
up here.
2012-10-22 21:14:26 +00:00
Konstantin Belousov
5050aa86cf Remove the support for using non-mpsafe filesystem modules.
In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.

The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.

Conducted and reviewed by:	attilio
Tested by:	pho
2012-10-22 17:50:54 +00:00
Eitan Adler
3d74f47b90 Correct the killpg(2) return values:
Return EPERM if processes were found but they
were unable to be signaled.

Return the first error from p_cansignal if no signal was successful.

Reviewed by:	jilles
Approved by:	cperciva
MFC after:	1 week
2012-10-22 03:43:02 +00:00
Eitan Adler
10950e4651 Colin acked the wrong diff originally. fixed version coming soon.
Approved by:	cperciva (implicit)
2012-10-22 03:36:44 +00:00
Eitan Adler
2a1c0e4d4e Correct the killpg(2) return values:
Return EPERM if processes were found but they
were unable to be signaled.

Return the first error from p_cansignal if no signal was successful.

Reviewed by:	jilles
Approved by:	cperciva
MFC after:	1 week
2012-10-22 03:34:43 +00:00
Eitan Adler
db702c59cf remove duplicate semicolons where possible.
Approved by:	cperciva
MFC after:	1 week
2012-10-22 03:00:37 +00:00
Andre Oppermann
dc00208ec4 Grammar fixes to r241781.
Submitted by:	alc
2012-10-20 19:38:22 +00:00
Andre Oppermann
2bdf61ca29 Hide the unfortunate named sysctl kern.ipc.somaxconn from sysctl -a
output and replace it with a new visible sysctl kern.ipc.acceptqueue
of the same functionality.  It specifies the maximum length of the
accept queue on a listen socket.

The old kern.ipc.somaxconn remains available for reading and writing
for compatibility reasons so that existing programs, scripts and
configurations continue to work.  There no plans to ever remove the
orginal and now hidden kern.ipc.somaxconn.
2012-10-20 12:53:14 +00:00
Andre Oppermann
1490de00a8 Tidy up somaxconn (accept queue limit) and related functions
and move it together into one place.
2012-10-20 10:51:32 +00:00
Andre Oppermann
4b62fe5b0b Move socket UMA zone initialization functionality together into
one place.
2012-10-19 12:16:29 +00:00
Andre Oppermann
cf8e6069e8 Move UMA socket zone initialization from uipc_domain.c to uipc_socket.c
into one place next to its other related functions to avoid confusion.
2012-10-19 10:15:32 +00:00
Andre Oppermann
d10733a8da Remove unnecessary includes from sosend_copyin() and fix
a couple of style issues.
2012-10-18 21:04:30 +00:00
Andre Oppermann
1d147759db Remove double-wrapping of #ifdef ZERO_COPY_SOCKETS within
zero copy specialized sosend_copyin() helper function.
2012-10-18 20:22:17 +00:00
Attilio Rao
2e564269d0 Disconnect non-MPSAFE SMBFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netsmb, which is a base
requirement for SMBFS.

In the while SMBFS regular users can use FUSE interface and smbnetfs
port to work with their SMBFS partitions.

Also, there are ongoing efforts by vendor to support in-kernel smbfs,
so there are good chances that it will get relinked once properly locked.

This is not targeted for MFC.
2012-10-18 12:04:56 +00:00
Attilio Rao
a42ac676f5 Disconnect non-MPSAFE NTFS from the build in preparation for dropping
GIANT from VFS. This code is particulary broken and fragile and other
in-kernel implementations around, found in other operating systems,
don't really seem clean and solid enough to be imported at all.
If someone wants to reconsider in-kernel NTFS implementation for
inclusion again, a fair effort for completely fixing and cleaning it
up is expected.

In the while NTFS regular users can use FUSE interface and ntfs-3g
port to work with their NTFS partitions.

This is not targeted for MFC.
2012-10-17 11:30:00 +00:00
Attilio Rao
e6116d5b8e Disconnect non-MPSAFE NWFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netncp, which is a base
requirement for NWFS.

In the possibility of a future maintenance of the code and later
readd to the FreeBSD base, maybe we should think about a better location
for netncp. I'm not entirely sure the / top location is actually right,
however I will let network people to comment on that more specifically.

This is not targeted for MFC.
2012-10-17 11:16:17 +00:00
Attilio Rao
55793cdccf Disconnect non-MPSAFE PORTALFS from the build in preparation for dropping
GIANT from VFS.

This is not targeted for MFC.
2012-10-16 09:59:10 +00:00
Attilio Rao
05e009c443 Disconnect non-MPSAFE HPFS from the build in preparation for dropping
GIANT from VFS.

This is not targeted for MFC.
2012-10-16 09:55:31 +00:00
Konstantin Belousov
36c6f3aaae Acquire the rangelock for truncate(2) as well.
Reported and reviewed by:	avg
Tested by:	pho
MFC after:	1 week
2012-10-15 18:15:18 +00:00
Konstantin Belousov
9b233e2307 Add a KPI to allow to reserve some amount of space in the numvnodes
counter, without actually allocating the vnodes. The supposed use of
the getnewvnode_reserve(9) is to reclaim enough free vnodes while the
code still does not hold any resources that might be needed during the
reclamation, and to consume the slack later for getnewvnode() calls
made from the innards. After the critical block is finished, the
caller shall free any reserve left, by getnewvnode_drop_reserve(9).

Reviewed by:	avg
Tested by:	pho
MFC after:	1 week
2012-10-14 19:43:37 +00:00
Alexander Motin
803a9b3efd panic() with reasonable message instead of returning zero frequency causing
division by zero later if event timer's minimal period is above one second.
For now it is just a theoretical possibility.

Found by:	Clang Static Analyzer
2012-10-10 19:46:46 +00:00
Attilio Rao
3a4730256a Add an unified macro to deny ability from the compiler to reorder
instruction loads/stores at its will.
The macro __compiler_membar() is currently supported for both gcc and
clang, but kernel compilation will fail otherwise.

Reviewed by:	bde, kib
Discussed with:	dim, theraven
MFC after:	2 weeks
2012-10-09 14:32:30 +00:00
Andriy Gapon
298fbd1605 cngetc: use cpu_spinwait to ease the cncheckc loop a tiny bit
Reviewed by:	julian
MFC after:	10 days
2012-10-06 19:50:23 +00:00
Andriy Gapon
c331c9703c ktrace/kern_exec: check p_tracecred instead of p_cred
.. when deciding whether to continue tracing across suid/sgid exec.
Otherwise if root ktrace-d an unprivileged process and the processed
exec-ed a suid program, then tracing didn't continue across exec.

Reviewed by:	bde, kib
MFC after:	22 days
2012-10-06 19:23:44 +00:00
Ed Schouten
6b1b791da6 Fix faulty error code handling in read(2) on TTYs.
When performing a non-blocking read(2), on a TTY while no data is
available, we should return EAGAIN. But if there's a modem disconnect,
we should return 0. Right now we only return 0 when doing a blocking
read, which is wrong.

MFC after:	1 month
2012-10-03 13:51:03 +00:00
Garrett Wollman
48b5c7410f Fix spelling of the function name in two assertion messages. 2012-10-02 18:38:05 +00:00
Eitan Adler
8dbce2a343 Provide a generic way to disable devices at boot time
PR:		kern/119202
Requested by:	peterj
Reviewed by:	sbruno, jhb
Approved by:	cperciva
MFC after:	1 week
2012-10-02 03:33:41 +00:00
Pawel Jakub Dawidek
55711729f3 - Enforce CAP_MKFIFO on mkfifoat(2), not on mknodat(2). Without this change
mkfifoat(2) was not restricted.
- Introduce CAP_MKNOD and enforce it on mknodat(2).

Sponsored by:	FreeBSD Foundation
MFC after:	2 weeks
2012-10-01 05:43:24 +00:00
Konstantin Belousov
877d24ac8a Fix the mis-handling of the VV_TEXT on the nullfs vnodes.
If you have a binary on a filesystem which is also mounted over by
nullfs, you could execute the binary from the lower filesystem, or
from the nullfs mount. When executed from lower filesystem, the lower
vnode gets VV_TEXT flag set, and the file cannot be modified while the
binary is active. But, if executed as the nullfs alias, only the
nullfs vnode gets VV_TEXT set, and you still can open the lower vnode
for write.

Add a set of VOPs for the VV_TEXT query, set and clear operations,
which are correctly bypassed to lower vnode.

Tested by:	pho (previous version)
MFC after:	2 weeks
2012-09-28 11:25:02 +00:00
Matthew D Fleming
fc8fdae0df Fix up kernel sources to be ready for a 64-bit ino_t.
Original code by:	Gleb Kurtsou
2012-09-27 23:30:49 +00:00
Pawel Jakub Dawidek
c8e781f6e0 Revert r240931, as the previous comment was actually in sync with POSIX.
I have to note that POSIX is simply stupid in how it describes O_EXEC/fexecve
and friends. Yes, not only inconsistent, but stupid.

In the open(2) description, O_RDONLY flag is described as:

	O_RDONLY	Open for reading only.

Taken from:

	http://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html

Note "for reading only". Not "for reading or executing"!

In the fexecve(2) description you can find:

	The fexecve() function shall fail if:

	[EBADF]
		The fd argument is not a valid file descriptor open for executing.

Taken from:

	http://pubs.opengroup.org/onlinepubs/9699919799/functions/exec.html

As you can see the function shall fail if the file was not open with O_EXEC!

And yet, if you look closer you can find this mess in the exec.html:

	Since execute permission is checked by fexecve(), the file description
	fd need not have been opened with the O_EXEC flag.

Yes, O_EXEC flag doesn't have to be specified after all. You can open a file
with O_RDONLY and you still be able to fexecve(2) it.
2012-09-27 16:43:23 +00:00
Mikolaj Golub
47813f5d94 Kernel and modules have "set_vnet" linker set, where virtualized
global variables are placed. When a module is loaded by link_elf
linker its variables from "set_vnet" linker set are copied to the
kernel "set_vnet" ("modspace") and all references to these variables
inside the module are relocated accordingly.

The issue is when a module is loaded that has references to global
variables from another, previously loaded module: these references are
not relocated so an invalid address is used when the module tries to
access the variable. The example is V_layer3_chain, defined in ipfw
module and accessed from ipfw_nat.

The same issue is with DPCPU variables, which use "set_pcpu" linker
set.

Fix this making the link_elf linker on a module load recognize
"external" DPCPU/VNET variables defined in the previously loaded
modules and relocate them accordingly. For this set_pcpu_list and
set_vnet_list are used, where the addresses of modules' "set_pcpu" and
"set_vnet" linker sets are stored.

Note, archs that use link_elf_obj (amd64) were not affected by this
issue.

Reviewed by:	jhb, julian, zec (initial version)
MFC after:	1 month
2012-09-27 14:55:15 +00:00
Konstantin Belousov
94cb35459d Make the updates of the tid ring buffer' head and tail pointers
explicit by moving them into separate statements from the buffer
element accesses.

Requested by:	jhb
MFC after:	3 days
2012-09-26 09:25:11 +00:00
Pawel Jakub Dawidek
28f865b0b1 Fix freebsd32_kmq_timedreceive() and freebsd32_kmq_timedsend() to use
getmq_read() and getmq_write() respectively, just like sys_kmq_timedreceive()
and sys_kmq_timedsend().

Sponsored by:	FreeBSD Foundation
MFC after:	2 weeks
2012-09-25 22:15:59 +00:00
Pawel Jakub Dawidek
8c706ce0d0 vn_write() always expects FOF_OFFSET flag, which is asserted at the begining,
so there is no need to check for it.

Sponsored by:	FreeBSD Foundation
MFC after:	2 weeks
2012-09-25 21:31:17 +00:00
Pawel Jakub Dawidek
3a038c4d68 We cannot open file for reading and executing (O_RDONLY | O_EXEC).
Well, in theory we can pass those two flags, because O_RDONLY is 0,
but we won't be able to read from a descriptor opened with O_EXEC.

Update the comment.

Sponsored by:	FreeBSD Foundation
MFC after:	2 weeks
2012-09-25 21:11:40 +00:00