18044 Commits

Author SHA1 Message Date
Chuck Silvers
11403bdeb4 vfs: fix rangelock range in vn_rdwr() for IO_APPEND
vn_rdwr() must lock the entire file range for IO_APPEND
just like vn_io_fault() does for O_APPEND.

Reviewed by:	kib, imp, mckusick
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D28008
2021-01-07 13:37:35 -08:00
Mateusz Guzik
f2b794e1e9 cache: unengrish the comment in previous commit
Reported by:	rpokala, brd
2021-01-06 23:46:05 +00:00
Mateusz Guzik
deabdc6868 cache: stop pre-checking seqc when starting the lookup
Tested by:	pho
2021-01-06 07:28:07 +00:00
Mateusz Guzik
71a6a0b545 cache: skip checking for spurious slashes if possible
Tested by:	pho
2021-01-06 07:28:06 +00:00
Mateusz Guzik
33f3e81df5 cache: combine fast path enabled status into one flag
Tested by:	pho
2021-01-06 07:28:06 +00:00
Mateusz Guzik
dbbbc07cc3 cache: split handling of 0 and non-0 error codes
Tested by:	pho
2021-01-06 07:07:24 +01:00
Mateusz Guzik
a1a8f8ada1 cache: deinline state handling
The intent is to reduce branchfest when finishing the lookup.

Tested by:	pho
2021-01-06 07:05:22 +01:00
Mateusz Guzik
05803be000 cache: stop setting cn_nameptr on entry as matches cn_pnbuf already
While here tidy up other asserts.
2021-01-06 07:03:41 +01:00
Mateusz Guzik
3814bea00a cache: drop the now spurious doomed check when crossing a mount point 2021-01-03 21:22:16 +00:00
Mateusz Guzik
33a195baf3 vfs: keep seqc unchanged as long as the vnode is accessible via SMR 2021-01-03 21:22:16 +00:00
Mark Johnston
214257da3a sendfile: Clear page pointers when handling a pager error
When INVARIANTS is configred, the sendfile_iodone() callback verifies
that pages attached to the sendfile header are wired, but we unwire all
such pages after a synchronous pager error, before calling
sendfile_iodone().

Reported by:	pho
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2021-01-03 11:50:31 -05:00
Mark Johnston
90f580b954 Ensure that dirent's d_off field is initialized
We have the d_off field in struct dirent for providing the seek offset
of the next directory entry.  Several filesystems were not initializing
the field, which ends up being copied out to userland.

Reported by:	Syed Faraz Abrar <faraz@elttam.com>
Reviewed by:	kib
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D27792
2021-01-03 11:50:31 -05:00
Mateusz Guzik
82397d7919 vfs: denote vnode being a mount point with VIRF_MOUNTPOINT
Reviewed by:	kib (previous version)
Differential Revision:	https://reviews.freebsd.org/D27794
2021-01-03 06:50:06 +00:00
Mateusz Guzik
3e506a67bb vfs: add v_irflag accessors
Reviewed by:	kib (previous version)
Differential Revision:	https://reviews.freebsd.org/D27793
2021-01-03 06:50:06 +00:00
Mateusz Guzik
51bf55fa6c cache: stop checkpointing cn_namelen
The variable is recomputed by regular lookup from the get go.
2021-01-03 06:50:06 +00:00
Mateusz Guzik
7220a10b5b cache: predict on no spurious slashes in cache_fpl_handle_root
This is a step towards speculatively not handling them.
2021-01-03 06:50:06 +00:00
Mateusz Guzik
30a2fc91fa cache: postpone NAME_MAX check as it may be unnecessary 2021-01-03 06:50:06 +00:00
Mateusz Guzik
eca899bd5d cache: remove spurious null check in sdt probe 2021-01-03 06:50:06 +00:00
Alan Somers
1868a91fac Regenerate syscall files after addition of aio_writev/aio_readv 2021-01-02 19:57:58 -07:00
Alan Somers
022ca2fc7f Add aio_writev and aio_readv
POSIX AIO is great, but it lacks vectored I/O functions. This commit
fixes that shortcoming by adding aio_writev and aio_readv. They aren't
part of the standard, but they're an obvious extension. They work just
like their synchronous equivalents pwritev and preadv.

It isn't yet possible to use vectored aiocbs with lio_listio, but that
could be added in the future.

Reviewed by:    jhb, kib, bcr
Relnotes:       yes
Differential Revision: https://reviews.freebsd.org/D27743
2021-01-02 19:57:58 -07:00
Jamie Gritton
b58a46347c jail: revert the attachment part of b4e87a632955
The change to kern_jail_set that was supposed to "also properly clean
up when attachment fails" didn't fix a memory leak but actually caused
a double free.  Back that part out, and leave the part that manages
allprison_lock state.
2020-12-31 19:55:49 -08:00
Mateusz Guzik
1365b5f86f cache: fold NCF_WHITE check into the rest
Tested by:	pho
2021-01-01 00:10:43 +00:00
Mateusz Guzik
d7c62d98c9 cache: call cache_fplookup_modifying in neg
Tested by:	pho
2021-01-01 00:10:43 +00:00
Mateusz Guzik
6fe7de1a25 cache: refactor cache_fpl_handle_root to fit the rest of the code better
Tested by:	pho
2021-01-01 00:10:43 +00:00
Mateusz Guzik
e17e01bd0e cache: refactor dot handling
Tested by:	pho
2021-01-01 00:10:43 +00:00
Mateusz Guzik
4651db56c7 cache: remove a branch from mount point checking
Tested by:	pho
2021-01-01 00:10:42 +00:00
Mateusz Guzik
0b5bd1afd8 cache: support lockless lookup of degenerate paths
Tested by:	pho
2021-01-01 00:10:42 +00:00
Mateusz Guzik
1d6eb97677 cache: save on branching when parsing the path by inserting a sentinel
Tested by:	pho
2021-01-01 00:10:42 +00:00
Mateusz Guzik
67297766b5 cache: hoist trailing slash and degenerate path handling out of the loop
Tested by:	pho
2021-01-01 00:10:42 +00:00
Mateusz Guzik
bb3a12f0e5 fd: inline pwd_get_smr
Tested by:	pho
2021-01-01 00:10:42 +00:00
John Baldwin
825d234144 Don't check P_INMEM in kdb_thr_*().
Not all debugger operations that enumerate threads require thread
stacks to be resident in memory to be useful.  Instead, push P_INMEM
checks (if needed) into callers.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27827
2020-12-31 16:01:12 -08:00
John Baldwin
9acce1c992 Enumerate processes via the pid hash table in kdb_thr_*().
Processes part way through exit1() are not included in allproc.  Using
allproc to enumerate processes prevented getting the stack trace of a
thread in this part of exit1() via ddb.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27826
2020-12-31 16:00:54 -08:00
John Baldwin
4e7d1b527c Add a proc_off_p_hash helper variable.
This is used by kernel debuggers to enumerate processes via the pid
hash table.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27825
2020-12-31 16:00:33 -08:00
John Baldwin
47877889f2 ddb ps: Use the pidhash to enumerate processes not in allproc.
Exiting processes that have been removed from allproc but are still
executing are not yet marked PRS_ZOMBIE, so they were not listed (for
example, if a thread panics during exit1()).  To detect these
processes, clear p_list.le_prev to NULL explicitly after removing a
process from the allproc list and check for this sentinel rather than
PRS_ZOMBIE when walking the pidhash.

While here, simplify the pidhash walk to use a single outer loop.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27824
2020-12-31 16:00:05 -08:00
Jamie Gritton
b4e87a6329 jail: Clean up allprison_lock handing in kern_jail_set
Keep explicit track of the allprison_lock state during the final part
of kern_jail_set, instead of deducing it from the JAIL_ATTACH flag.

Also properly clean up when the attachment fails, fixing a long-
standing (though minor) memory leak.
2020-12-31 15:18:43 -08:00
Mateusz Guzik
0c09f4b0cc cache: work around corner case of dvp == tvp in cache_fplookup_final_modifying
Fixes a panic where the kernel would unlock an unheld lock coming from
rename looking up "foo/." as the source.

Reported by:	markj (syzkaller)
2020-12-28 21:38:20 +00:00
Mateusz Guzik
4ab7d9f484 cache: reduce engrish in previous commit 2020-12-28 02:05:30 +00:00
Mateusz Guzik
0714f921cd cache: save on some branching in common case mount point traversal 2020-12-28 01:53:28 +00:00
Mateusz Guzik
8c9d74634a vfs: stop open-coding setting WILLBEDIR flag 2020-12-28 01:53:27 +00:00
Mateusz Guzik
002e18eb7f vfs: add FAILIFEXISTS flag
Both FreeBSD and Linux mkdir -p walk the tree up ignoring any EEXIST on
the way and both are used a lot when building respective kernels.

This poses a problem as spurious locking avoidably interferes with
concurrent operations like getdirentries on affected directories.

Work around the problem by adding FAILIFEXISTS flag. In case of lockless
lookup this manages to avoid any work to begin with, there is no speed
up for the locked case but perhaps this can be augmented later on.

For simplicity the only supported semantics are as used by mkdir.

Reviewed by:	kib (previous version)
Differential Revision:	https://reviews.freebsd.org/D27789
2020-12-28 01:53:27 +00:00
Mateusz Guzik
ff97bc034f cache: simplify lockless dot lookups 2020-12-28 01:53:27 +00:00
Mateusz Guzik
abd7ded451 cache: modification and last entry filling support in lockless lookup v2
The previous patch failed to set the ISDOTDOT flag when appropriate,
which in turn fail to properly handle degenerate lookups.

While here sprinkle some extra assertions.

Tested by:	pho (previous version)
2020-12-27 21:03:18 +00:00
Mateusz Guzik
623daa69f9 cache: assert internal flags are not passed by namei 2020-12-27 19:49:24 +00:00
Mateusz Guzik
a1fc1f10c6 Revert "cache: modification and last entry filling support in lockless lookup"
This reverts commit 6dbb07ed6872ae7988b9b705e322c94658eba6d1.

Some ports unreliably fail to build with rmdir getting ENOTEMPTY.
2020-12-27 19:02:29 +00:00
Mateusz Guzik
6dbb07ed68 cache: modification and last entry filling support in lockless lookup
Tested by:	pho (previous version)
2020-12-27 17:22:25 +00:00
Konstantin Belousov
9dd48b87e6 Regen. 2020-12-27 12:57:27 +02:00
Konstantin Belousov
7a202823aa Expose eventfd in the native API/ABI using a new __specialfd syscall
eventfd is a Linux system call that produces special file descriptors
for event notification. When porting Linux software, it is currently
usually emulated by epoll-shim on top of kqueues.  Unfortunately, kqueues
are not passable between processes.  And, as noted by the author of
epoll-shim, even if they were, the library state would also have to be
passed somehow.  This came up when debugging strange HW video decode
failures in Firefox.  A native implementation would avoid these problems
and help with porting Linux software.

Since we now already have an eventfd implementation in the kernel (for
the Linuxulator), it's pretty easy to expose it natively, which is what
this patch does.

Submitted by:   greg@unrelenting.technology
Reviewed by:    markj (previous version)
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D26668
2020-12-27 12:57:26 +02:00
Jamie Gritton
7f4e724829 jail: add a missing lock around an osd_jail_call().
allprison_lock should be at least held shared when jail OSD methods
are called.  Add a shared lock around one such call where that wasn't
the case.

In another such call, change an exclusive lock grab to be shared in
what is likely the more common case.
2020-12-26 20:49:30 -08:00
Jamie Gritton
0fe74ae624 jail: Consistently handle the pr_allow bitmask
Return a boolean (i.e. 0 or 1) from prison_allow, instead of the flag
value itself, which is what sysctl expects.

Add prison_set_allow(), which can set or clear a permission bit, and
propagates cleared bits down to child jails.

Use prison_allow() and prison_set_allow() in the various jail.allow.*
sysctls, and others that depend on thoe permissions.

Add locking around checking both pr_allow and pr_enforce_statfs in
prison_priv_check().
2020-12-26 20:25:02 -08:00
Mark Johnston
26b23f07fb sendfile: Ensure that sfio->npages is initialized
We initialize sfio->npages only when some I/O is required to satisfy the
request.  However, sendfile_iodone() contains an INVARIANTS-only check
that references sfio->npages, and this check is executed even if no I/O
is performed, so the check may use an uninitialized value.

Fix the problem by initializing sfio->npages earlier.  Note that
sendfile_swapin() always initializes the page array.  In some rare cases
we need to trim the page array so ensure that sfio->npages gets updated
accordingly.

Reported by:		syzkaller (with KASAN)
Reviewed by:		kib
Sponsored by:		The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D27726
2020-12-26 16:07:40 -05:00