211231 Commits

Author SHA1 Message Date
kib
564e88d499 Do not allow creation of the dirty buffers for the dead buffer
objects, i.e. for buffer objects which vnode was reclaimed.  Buffer
cache cannot write such buffers.  Return the error and discard the
buffer immediately on write attempt.

BO_DIRTY now always set during vnode reclamation, since it is used not
only for the INVARIANTS checks.  Do allow placement of the clean
buffers on dead bufobj list, otherwise filesystems cannot use bufcache
at all after the devvp reclaim.

Reported and tested by:	trasz
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2015-07-11 11:21:56 +00:00
jmg
4be9e0b121 some additional improvements to the documentation...
Sponsored by:	Netflix, Inc.
2015-07-11 04:20:56 +00:00
jmg
ec1fe94d10 Complete the move that was started w/ r263218.. For some reason I
didn't delete the files, so that means we need to bring the changes in
r282726 to the correct files..

make tinderbox completed with this patch...
2015-07-11 03:12:34 +00:00
pjd
1f882f57d3 Spoil even can happen for some time now even on providers opened exclusively
(on the media change event). Update GELI to handle that situation.

PR:		201185
Submitted by:	Matthew D. Fuller
2015-07-10 19:27:19 +00:00
luigi
c112159436 assorted algorithmic fixes from Paolo Valente (one of my qfq coauthors):
- use 1ULL to avoid shift truncations
- recompute the sum of weight dynamically to provide better fairness
- fix an erroneous constant in the computation of the slot
- preserve timestamp correctness when the old timestamp is stale.
2015-07-10 19:24:36 +00:00
luigi
9c7bfd6b7d one more warning suppression when compiling the test code in userspace. 2015-07-10 19:18:49 +00:00
luigi
5a4c84322b add code to compute fairness indexes;
cleanups to remove compile warnings.
2015-07-10 18:10:40 +00:00
luigi
65802f0a4f staticize functions only used in netmap.c
(detected by jenkins run with gcc 4.9)

Update documentation on the use of netmap_priv_d,
rename the refcount and use the same structure in
FreeBSD and linux

No functional changes.
2015-07-10 16:05:24 +00:00
ed
62ec0a1901 Add missing const keyword to kern_sigaction()'s 'act' parameter.
This structure is not modified by the function. Also add const to
sigact_flag_test(), as it is called by kern_sigaction().
2015-07-10 14:39:46 +00:00
mjg
2efe5a9a7a fd: further cleanup of kern_dup
- make mode enum start from 0 so that the assertion covers all cases [1]
- rename prefix _CLOEXEC flag with _FLAG
- postpone fhold on the old file descriptor, which eliminates the need to fdrop
  in error cases.
- fixup FDDUP_FCNTL check missed in the previous commit

This removes 'fp == oldfde->fde_file' assertion which had little value. kern_dup
only calls fd-related functions which cannot drop the lock or a whole lot of
races would be introduced.

Noted by: kib [1]
2015-07-10 13:54:03 +00:00
mjg
b3aa72d2a3 fd: split kern_dup flags argument into actual flags and a mode
Tidy up the code inside to switch on the mode.
2015-07-10 11:01:30 +00:00
kib
04c0ab7814 Convert between abridged (from FXSAVE) and unabridged (from FSAVE)
versions of the x87 tags.  The conversion is naive, used abridged tag
is converted to valid unabridged, without additional checks for zero
and special values.

Noted by:	bde
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-07-10 09:20:13 +00:00
kib
a4e4a26325 Duplicate the copyright from the i386/i386/machdep.c into
i386/include/frame.h after a code was moved from machdep.c to frame.h
in r284925.

Use include guards style similar to other guards.

Noted by:	bde
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2015-07-10 09:15:06 +00:00
kib
8b3d38af03 Change the mb() use in the sched_ult tdq_notify() and sched_idletd()
to more C11-ish atomic_thread_fence_seq_cst().

Note that on PowerPC, which currently uses lwsync for mb(), the change
actually fixes the missed store/load barrier, intended by r271604 [*].

Reviewed by:	alc
Noted by:	alc [*]
Sponsored by:	The FreeBSD Foundation
MFC after:	3 weeks
2015-07-10 08:54:12 +00:00
andrew
87fffd8ba1 Add support for makecontext. This supports up to 8 arguments as this
simplifies the code, these can be passed in registers.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-10 08:36:22 +00:00
luigi
41752fc79f add netmap dependency when compiled as a module 2015-07-10 07:13:14 +00:00
ed
790c476c1a Let listen() return EDESTADDRREQ when not bound.
We currently return EINVAL when calling listen() on a UNIX socket that
has not been bound to a pathname. If my interpretation of POSIX is
correct, we should return EDESTADDRREQ: "The socket is not bound to a
local address, and the protocol does not support listening on an unbound
socket."

Return EDESTADDRREQ instead when not bound and not connected.

Differential Revision:	https://reviews.freebsd.org/D3038
Reviewed by:	gnn, network
2015-07-10 06:47:14 +00:00
luigi
c354cad8fd Sync netmap sources with the version in our private tree.
This commit contains large contributions from Giuseppe Lettieri and
Stefano Garzarella, is partly supported by grants from Verisign and Cisco,
and brings in the following:

- fix zerocopy monitor ports and introduce copying monitor ports
  (the latter are lower performance but give access to all traffic
  in parallel with the application)

- exclusive open mode, useful to implement solutions that recover
  from crashes of the main netmap client (suggested by Patrick Kelsey)

- revised memory allocator in preparation for the 'passthrough mode'
  (ptnetmap) recently presented at bsdcan. ptnetmap is described in
        S. Garzarella, G. Lettieri, L. Rizzo;
        Virtual device passthrough for high speed VM networking,
        ACM/IEEE ANCS 2015, Oakland (CA) May 2015
        http://info.iet.unipi.it/~luigi/research.html

- fix rx CRC handing on ixl

- add module dependencies for netmap when building drivers as modules

- minor simplifications to device-specific routines (*txsync, *rxsync)

- general code cleanup (remove unused variables, introduce macros
  to access rings and remove duplicate code,

Applications do not need to be recompiled, unless of course
they want to use the new features (monitors and exclusive open).

Those willing to try this code on stable/10 can just update the
sys/dev/netmap/*, sys/net/netmap* with the version in HEAD
and apply the small patches to individual device drivers.

MFC after:	1 month
Sponsored by:	(partly) Verisign, Cisco
2015-07-10 05:51:36 +00:00
luigi
0f9bc3ce0c rev.284898 removed _SHLIBDIRPREFIX so we need to reconstruct its value
to properly locate libraries created in the buildworld phase.
2015-07-10 05:07:18 +00:00
gnn
4d0b1feb1d Summary: Fix LINT build. The names of the new AES modes were not
correctly used under the REGRESSION kernel option.
2015-07-10 02:23:50 +00:00
dim
0a09d271d9 Fix swapped copyin(9) arguments in cxgb's iwch_arm_cq() function.
Detected by clang 3.7.0 with the warning:

sys/dev/cxgb/ulp/iw_cxgb/iw_cxgb_provider.c:309:18: error: variable
'rptr' is uninitialized when used here [-Werror,-Wuninitialized]
                chp->cq.rptr = rptr;
                               ^~~~

MFC after:	1 week
2015-07-09 22:13:23 +00:00
oshogbo
3f0ef50892 Rename zfs nvpair files to not colidate with our nvlist.
PR:		201356
Approved by:	pjd (mentor)
2015-07-09 21:53:40 +00:00
andrew
6a4a0db965 Remove checks for __ARM_EABI__, we only build for EABI now.
Sponsored by:	ABT Systems Ltd
2015-07-09 21:02:40 +00:00
andrew
b85d394d0e Add support for __aeabi_memclr4, clang 3.7 calls it.
Sponsored by:	ABT Systems Ltd
2015-07-09 20:54:38 +00:00
gnn
5213809aeb Add support for AES modes to IPSec. These modes work both in software only
mode and with hardware support on systems that have AESNI instructions.

Differential Revision:	D2936
Reviewed by:	jmg, eri, cognet
Sponsored by:	Rubicon Communications (Netgate)
2015-07-09 18:16:35 +00:00
andrew
621d08aeda Clear the carry bit on the saved program state register when asked to
clear the return value, it's used to indicate an error.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-09 17:26:56 +00:00
gjb
abb2957ecd Document r285329, OpenSSL update to 1.0.1p.
Sponsored by:	The FreeBSD Foundation
2015-07-09 17:24:54 +00:00
mjg
03b75d4638 vfs: cosmetic changes to namei and namei_handle_root
- don't initialize cnp during declaration
- don't test error/!error, compare to 0 instead
2015-07-09 17:17:26 +00:00
jkim
ce8a666092 Merge OpenSSL 1.0.1p. 2015-07-09 17:07:45 +00:00
jkim
e5911a7a89 Import OpenSSL 1.0.1p. 2015-07-09 16:41:34 +00:00
mjg
2c26f2224b vfs: simplify error handling in namei
The logic is reorganised so that there is one exit point prior to the
lookup loop. This is an intermediate step to making audit logging
functions use found vnode instead of translating ni_dirfd on their own.

ni_startdir validation is removed. The only in-tree consumer is nfs
which already makes sure it is a directory.

Reviewed by:	kib
2015-07-09 16:32:58 +00:00
eri
faf2b7a96c Correct issue presented in r285051,
apparently neither clang nor gcc complain about this.
But clang intis the var to NULL correctly while gcc on at least mips does not.
Correct the undefined behavior by initializing the variable properly.

PR:		201371
Differential Revision:	 https://reviews.freebsd.org/D3036
Reviewed by:	gnn
Approved by:	gnn(mentor)
2015-07-09 16:28:36 +00:00
jmg
9b2d45a850 increase buffer size to significantly increase performance...
see:
https://docs.freebsd.org/cgi/mid.cgi?20150513080342.GE37063@funkthat.com

for benchmarks...
2015-07-09 16:13:05 +00:00
ed
8cbb0879ba Add implementations for some of the CloudABI file descriptor system calls.
All of the CloudABI system calls that operate on file descriptors of an
arbitrary type are prefixed with fd_. This change adds wrappers for
most of these system calls around their FreeBSD equivalents.

The dup2() system call present on CloudABI deviates from POSIX, in the
sense that it can only be used to replace existing file descriptor. It
cannot be used to create new ones. The reason for this is that this is
inherently thread-unsafe. Furthermore, there is no need on CloudABI to
use fixed file descriptor numbers. File descriptors 0, 1 and 2 have no
special meaning.

This change exposes the kern_dup() through <sys/syscallsubr.h> and puts
the FDDUP_* flags in <sys/filedesc.h>. It then adds a new flag,
FDDUP_MUSTREPLACE to force that file descriptors are replaced -- not
allocated.

Differential Revision:	https://reviews.freebsd.org/D3035
Reviewed by:	mjg
2015-07-09 16:07:01 +00:00
mjg
d4c928e3e7 fd: prepare do_dup for being exported
- rename it to kern_dup.
- prefix flags with FD
- assert that correct flags were passed
2015-07-09 15:19:45 +00:00
mjg
5c01a53df7 vfs: avoid spurious vref/vrele for absolute lookups
namei used to vref fd_cdir, which was immediatley vrele'd on entry to
the loop.

Check for absolute lookup and vref the right vnode the first time.

Reviewed by:	kib
2015-07-09 15:06:58 +00:00
mjg
5bbdbadc3d vfs: plug a use-after-free of fd_rdir in namei
fd_rdir vnode was stored in ni_rootdir without refing it in any way,
after which the filedsc lock was being dropped.

The vnode could have been freed by mountcheckdirs or another thread doing
chroot.

VREF the vnode while the lock is held.

Reviewed by:	kib
MFC after:	1 week
2015-07-09 15:06:24 +00:00
bapt
f97075fb60 Do not try to set password on group if the group is added as a consequence of
of creating a user (regression from r285136)

Reported by:	Fabian Keil <fk@fabiankeil.de>
2015-07-09 14:14:44 +00:00
andrew
561cbb7709 Add support for SMP. This uses the FDT data to find the CPUs to start on,
and psci to start them. I expect ACPI support to be added later.

This has been tested on qemu with 2 cpus as that is the current value of
MAXCPUS. This is expected to be increased in the future as FreeBSD has
been tested on 48 cores on the Cavium ThunderX hardware.

Partially based on a patch from Robin Randhawa from ARM.

Approved by:	ABT Systems Ltd
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D3024
2015-07-09 13:23:29 +00:00
andrew
e561c6cbca Add logging of synchronous exceptions.
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-09 13:07:12 +00:00
andrew
802b17ef78 Add the definition of the shareable bits in the pagetables
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-09 12:56:09 +00:00
andrew
1bff17935d Clean up the types used in <machine/ucontext.h> on arm64. As some ports
include this file without first including the headers needed for uint32_t
and the like use the __foo type.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
2015-07-09 12:51:50 +00:00
ed
ed913b3a6c Don't clobber td->td_retval[0] in proc_reap().
While writing tests for CloudABI, I noticed that close() on process
descriptors returns the process ID of the child process. This is
interesting, as close() is only allowed to return 0 or -1. It turns out
that we clobber td->td_retval[0] in proc_reap(), so that wait*()
properly returns the process ID.

Change proc_reap() to leave td->td_retval[0] alone. Set the return value
in kern_wait6() instead, by keeping track of the PID before we
(potentially) reap the process.

Differential Revision:	https://reviews.freebsd.org/D3032
Reviewed by:	kib
2015-07-09 12:04:45 +00:00
zbb
99ff382570 Rework CPU identification on ARM64
This commit reworks the code responsible for identification of
the CPUs during runtime.
It is necessary to provide a way for workarounds and erratums
to be applied only for certain HW versions.

The copy of MIDR is now stored in pcpu to provide a fast and
convenient way for assambly code to read it (pcpu is used quite often
so there is a chance it's inside the cache).
The MIDR is also better way of identification than using user-friendly
cpu_desc structure, because it can be compiled into comparision of
single u32 with only one access to the memory - this is crucial
for some erratums which are called from performance-critical
places.

Changes in cpu_identify makes this function safe to be called
on non-boot CPUs.

New function CPU_MATCH was implemented which returns boolean
value based on mathing masked MIDR with chip identification.
Example of usage:

printf("is thunder: %d\n", CPU_MATCH(CPU_IMPL_MASK | CPU_PART_MASK,
        CPU_IMPL_CAVIUM, CPU_PART_THUNDER, 0, 0));
printf("is generic: %d\n", CPU_MATCH(CPU_IMPL_MASK | CPU_PART_MASK,
        CPU_IMPL_ARM, CPU_PART_FOUNDATION, 0, 0));

Reviewed by:   andrew
Obtained from: Semihalf
Sponsored by:  The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3030
2015-07-09 11:32:29 +00:00
kib
58e696649e Cover a race between doselwakeup() and selfdfree(). If doselwakeup()
loop finds the selfd entry and clears its sf_si pointer, which is
handled by selfdfree() in parallel, NULL sf_si makes selfdfree() free
the memory.  The result is the race and accesses to the freed memory.

Refcount the selfd ownership.  One reference is for the sf_link
linkage, which is unconditionally dereferenced by selfdfree().
Another reference is for sf_threads, both selfdfree() and
doselwakeup() race to deref it, the winner unlinks and than frees the
selfd entry.

Reported by:	Larry Rosenman <ler@lerctr.org>
Tested by:	Larry Rosenman <ler@lerctr.org>, pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2015-07-09 09:22:21 +00:00
ed
6dc0345fb0 Add forward declaration of struct thread.
This structure is used in some of the functions in this header, but we
don't depend on any header that pulls it i.
2015-07-09 07:31:40 +00:00
ed
a69356d93d Generate CloudABI system call table with proper $FreeBSD$ tags. 2015-07-09 07:21:33 +00:00
ed
af8adb5b79 Import the CloudABI datatypes and create a system call table.
CloudABI is a pure capability-based runtime environment for UNIX. It
works similar to Capsicum, except that processes already run in
capabilities mode on startup. All functionality that conflicts with this
model has been omitted, making it a compact binary interface that can be
supported by other operating systems without too much effort.

CloudABI is 'secure by default'; the idea is that it should be safe to
run arbitrary third-party binaries without requiring any explicit
hardware virtualization (Bhyve) or namespace virtualization (Jails). The
rights of an application are purely determined by the set of file
descriptors that you grant it on startup.

The datatypes and constants used by CloudABI's C library (cloudlibc) are
defined in separate files called syscalldefs_mi.h (pointer size
independent) and syscalldefs_md.h (pointer size dependent). We import
these files in sys/contrib/cloudabi and wrap around them in
cloudabi*_syscalldefs.h.

We then add stubs for all of the system calls in sys/compat/cloudabi or
sys/compat/cloudabi64, depending on whether the system call depends on
the pointer size. We only have nine system calls that depend on the
pointer size. If we ever want to support 32-bit binaries, we can simply
add sys/compat/cloudabi32 and implement these nine system calls again.

The next step is to send in code reviews for the individual system call
implementations, but also add a sysentvec, to allow CloudABI executabled
to be started through execve().

More information about CloudABI:
- GitHub: https://github.com/NuxiNL/cloudlibc
- Talk at BSDCan: https://www.youtube.com/watch?v=SVdF84x1EdA

Differential Revision:	https://reviews.freebsd.org/D2848
Reviewed by:	emaste, brooks
Obtained from:	https://github.com/NuxiNL/freebsd
2015-07-09 07:20:15 +00:00
pkelsey
b23b1a18f0 MFV r285292:
Merge upstream fix to eliminate build-breaking gcc warnings of no
importance.

commit: cab33b7a0acba7d2268a23c4383be6167106e549

Update ND_TTEST2 to fix issue 443

Add IS_NOT_NEGATIVE macro.
Avoid these warnings:
- comparison of unsigned expression >= 0 is always true [-Wtype-limits],
- comparison is always true due to limited range of data type [-Wtype-limits].

Reviewed by: adrian
Approved by: jmallett (mentor)
MFC after: 1 month
2015-07-08 23:57:58 +00:00
jmg
d374814126 upon further examination, it turns out that _unregister_all already
provides the guarantee that no threads will be in the _newsession code..
This is provided by the CRYPTODRIVER lock...  This makes the pause
unneeded...
2015-07-08 22:48:41 +00:00