Commit Graph

18052 Commits

Author SHA1 Message Date
Alan Somers
b3286afae3 Reallocate syscall numbers for aio_writev and aio_readv
The originally chosen numbers interfere with downstream projects'
syscalls.  Move them to the end of the syscall table instead.

Reported by:	jrtc27
Reviewed by:	brooks
MFC-With:	022ca2fc7f
Differential Revision:	022ca2fc7f
2021-01-07 19:49:27 -07:00
Thomas Munro
801ac943ea aio_fsync(2): Support O_DSYNC.
aio_fsync(O_DSYNC, ...) is the asynchronous version of fdatasync(2).

Reviewed by: kib, asomers, jhb
Differential Review: https://reviews.freebsd.org/D25071
2021-01-08 13:15:56 +13:00
Thomas Munro
a5e284038e open(2): Add O_DSYNC flag.
POSIX O_DSYNC means that writes include an implicit fdatasync(2), just
as O_SYNC implies fsync(2).

VOP_WRITE() functions that understand the new IO_DATASYNC flag can act
accordingly, but we'll still pass down IO_SYNC so that file systems that
don't understand it will continue to provide the stronger O_SYNC
behaviour.

Flag also applies to fcntl(2).

Reviewed by: kib, delphij
Differential Revision: https://reviews.freebsd.org/D25090
2021-01-08 13:15:56 +13:00
Mateusz Guzik
71bd18d373 fd: use seqc_read_notmodify when translating fds 2021-01-07 23:30:04 +00:00
Mateusz Guzik
20ac5cda96 fd: make fd/fp mandatory
They are both always passed anyway.
2021-01-07 23:30:04 +00:00
Mateusz Guzik
fee405e057 cache: stop checkpointing cn_flags
They are only modified, if ever, for the last component.
2021-01-07 23:29:52 +00:00
Mateusz Guzik
ac7715471c cache: stop checkpointing cn_nameptr
For aborts cn_nameptr is the same as cn_pnbuf. For partial results
the same cn_nameptr is to be used.
2021-01-07 23:29:38 +00:00
Mateusz Guzik
0f1fc3a31f cache: stop manipulating pathlen
It is a copy-pasto from regular lookup. Add debug to ensure the result
is the same.
2021-01-07 23:26:53 +00:00
Chuck Silvers
11403bdeb4 vfs: fix rangelock range in vn_rdwr() for IO_APPEND
vn_rdwr() must lock the entire file range for IO_APPEND
just like vn_io_fault() does for O_APPEND.

Reviewed by:	kib, imp, mckusick
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D28008
2021-01-07 13:37:35 -08:00
Mateusz Guzik
f2b794e1e9 cache: unengrish the comment in previous commit
Reported by:	rpokala, brd
2021-01-06 23:46:05 +00:00
Mateusz Guzik
deabdc6868 cache: stop pre-checking seqc when starting the lookup
Tested by:	pho
2021-01-06 07:28:07 +00:00
Mateusz Guzik
71a6a0b545 cache: skip checking for spurious slashes if possible
Tested by:	pho
2021-01-06 07:28:06 +00:00
Mateusz Guzik
33f3e81df5 cache: combine fast path enabled status into one flag
Tested by:	pho
2021-01-06 07:28:06 +00:00
Mateusz Guzik
dbbbc07cc3 cache: split handling of 0 and non-0 error codes
Tested by:	pho
2021-01-06 07:07:24 +01:00
Mateusz Guzik
a1a8f8ada1 cache: deinline state handling
The intent is to reduce branchfest when finishing the lookup.

Tested by:	pho
2021-01-06 07:05:22 +01:00
Mateusz Guzik
05803be000 cache: stop setting cn_nameptr on entry as matches cn_pnbuf already
While here tidy up other asserts.
2021-01-06 07:03:41 +01:00
Mateusz Guzik
3814bea00a cache: drop the now spurious doomed check when crossing a mount point 2021-01-03 21:22:16 +00:00
Mateusz Guzik
33a195baf3 vfs: keep seqc unchanged as long as the vnode is accessible via SMR 2021-01-03 21:22:16 +00:00
Mark Johnston
214257da3a sendfile: Clear page pointers when handling a pager error
When INVARIANTS is configred, the sendfile_iodone() callback verifies
that pages attached to the sendfile header are wired, but we unwire all
such pages after a synchronous pager error, before calling
sendfile_iodone().

Reported by:	pho
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
2021-01-03 11:50:31 -05:00
Mark Johnston
90f580b954 Ensure that dirent's d_off field is initialized
We have the d_off field in struct dirent for providing the seek offset
of the next directory entry.  Several filesystems were not initializing
the field, which ends up being copied out to userland.

Reported by:	Syed Faraz Abrar <faraz@elttam.com>
Reviewed by:	kib
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D27792
2021-01-03 11:50:31 -05:00
Mateusz Guzik
82397d7919 vfs: denote vnode being a mount point with VIRF_MOUNTPOINT
Reviewed by:	kib (previous version)
Differential Revision:	https://reviews.freebsd.org/D27794
2021-01-03 06:50:06 +00:00
Mateusz Guzik
3e506a67bb vfs: add v_irflag accessors
Reviewed by:	kib (previous version)
Differential Revision:	https://reviews.freebsd.org/D27793
2021-01-03 06:50:06 +00:00
Mateusz Guzik
51bf55fa6c cache: stop checkpointing cn_namelen
The variable is recomputed by regular lookup from the get go.
2021-01-03 06:50:06 +00:00
Mateusz Guzik
7220a10b5b cache: predict on no spurious slashes in cache_fpl_handle_root
This is a step towards speculatively not handling them.
2021-01-03 06:50:06 +00:00
Mateusz Guzik
30a2fc91fa cache: postpone NAME_MAX check as it may be unnecessary 2021-01-03 06:50:06 +00:00
Mateusz Guzik
eca899bd5d cache: remove spurious null check in sdt probe 2021-01-03 06:50:06 +00:00
Alan Somers
1868a91fac Regenerate syscall files after addition of aio_writev/aio_readv 2021-01-02 19:57:58 -07:00
Alan Somers
022ca2fc7f Add aio_writev and aio_readv
POSIX AIO is great, but it lacks vectored I/O functions. This commit
fixes that shortcoming by adding aio_writev and aio_readv. They aren't
part of the standard, but they're an obvious extension. They work just
like their synchronous equivalents pwritev and preadv.

It isn't yet possible to use vectored aiocbs with lio_listio, but that
could be added in the future.

Reviewed by:    jhb, kib, bcr
Relnotes:       yes
Differential Revision: https://reviews.freebsd.org/D27743
2021-01-02 19:57:58 -07:00
Jamie Gritton
b58a46347c jail: revert the attachment part of b4e87a6329
The change to kern_jail_set that was supposed to "also properly clean
up when attachment fails" didn't fix a memory leak but actually caused
a double free.  Back that part out, and leave the part that manages
allprison_lock state.
2020-12-31 19:55:49 -08:00
Mateusz Guzik
1365b5f86f cache: fold NCF_WHITE check into the rest
Tested by:	pho
2021-01-01 00:10:43 +00:00
Mateusz Guzik
d7c62d98c9 cache: call cache_fplookup_modifying in neg
Tested by:	pho
2021-01-01 00:10:43 +00:00
Mateusz Guzik
6fe7de1a25 cache: refactor cache_fpl_handle_root to fit the rest of the code better
Tested by:	pho
2021-01-01 00:10:43 +00:00
Mateusz Guzik
e17e01bd0e cache: refactor dot handling
Tested by:	pho
2021-01-01 00:10:43 +00:00
Mateusz Guzik
4651db56c7 cache: remove a branch from mount point checking
Tested by:	pho
2021-01-01 00:10:42 +00:00
Mateusz Guzik
0b5bd1afd8 cache: support lockless lookup of degenerate paths
Tested by:	pho
2021-01-01 00:10:42 +00:00
Mateusz Guzik
1d6eb97677 cache: save on branching when parsing the path by inserting a sentinel
Tested by:	pho
2021-01-01 00:10:42 +00:00
Mateusz Guzik
67297766b5 cache: hoist trailing slash and degenerate path handling out of the loop
Tested by:	pho
2021-01-01 00:10:42 +00:00
Mateusz Guzik
bb3a12f0e5 fd: inline pwd_get_smr
Tested by:	pho
2021-01-01 00:10:42 +00:00
John Baldwin
825d234144 Don't check P_INMEM in kdb_thr_*().
Not all debugger operations that enumerate threads require thread
stacks to be resident in memory to be useful.  Instead, push P_INMEM
checks (if needed) into callers.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27827
2020-12-31 16:01:12 -08:00
John Baldwin
9acce1c992 Enumerate processes via the pid hash table in kdb_thr_*().
Processes part way through exit1() are not included in allproc.  Using
allproc to enumerate processes prevented getting the stack trace of a
thread in this part of exit1() via ddb.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27826
2020-12-31 16:00:54 -08:00
John Baldwin
4e7d1b527c Add a proc_off_p_hash helper variable.
This is used by kernel debuggers to enumerate processes via the pid
hash table.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27825
2020-12-31 16:00:33 -08:00
John Baldwin
47877889f2 ddb ps: Use the pidhash to enumerate processes not in allproc.
Exiting processes that have been removed from allproc but are still
executing are not yet marked PRS_ZOMBIE, so they were not listed (for
example, if a thread panics during exit1()).  To detect these
processes, clear p_list.le_prev to NULL explicitly after removing a
process from the allproc list and check for this sentinel rather than
PRS_ZOMBIE when walking the pidhash.

While here, simplify the pidhash walk to use a single outer loop.

Reviewed by:	kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27824
2020-12-31 16:00:05 -08:00
Jamie Gritton
b4e87a6329 jail: Clean up allprison_lock handing in kern_jail_set
Keep explicit track of the allprison_lock state during the final part
of kern_jail_set, instead of deducing it from the JAIL_ATTACH flag.

Also properly clean up when the attachment fails, fixing a long-
standing (though minor) memory leak.
2020-12-31 15:18:43 -08:00
Mateusz Guzik
0c09f4b0cc cache: work around corner case of dvp == tvp in cache_fplookup_final_modifying
Fixes a panic where the kernel would unlock an unheld lock coming from
rename looking up "foo/." as the source.

Reported by:	markj (syzkaller)
2020-12-28 21:38:20 +00:00
Mateusz Guzik
4ab7d9f484 cache: reduce engrish in previous commit 2020-12-28 02:05:30 +00:00
Mateusz Guzik
0714f921cd cache: save on some branching in common case mount point traversal 2020-12-28 01:53:28 +00:00
Mateusz Guzik
8c9d74634a vfs: stop open-coding setting WILLBEDIR flag 2020-12-28 01:53:27 +00:00
Mateusz Guzik
002e18eb7f vfs: add FAILIFEXISTS flag
Both FreeBSD and Linux mkdir -p walk the tree up ignoring any EEXIST on
the way and both are used a lot when building respective kernels.

This poses a problem as spurious locking avoidably interferes with
concurrent operations like getdirentries on affected directories.

Work around the problem by adding FAILIFEXISTS flag. In case of lockless
lookup this manages to avoid any work to begin with, there is no speed
up for the locked case but perhaps this can be augmented later on.

For simplicity the only supported semantics are as used by mkdir.

Reviewed by:	kib (previous version)
Differential Revision:	https://reviews.freebsd.org/D27789
2020-12-28 01:53:27 +00:00
Mateusz Guzik
ff97bc034f cache: simplify lockless dot lookups 2020-12-28 01:53:27 +00:00
Mateusz Guzik
abd7ded451 cache: modification and last entry filling support in lockless lookup v2
The previous patch failed to set the ISDOTDOT flag when appropriate,
which in turn fail to properly handle degenerate lookups.

While here sprinkle some extra assertions.

Tested by:	pho (previous version)
2020-12-27 21:03:18 +00:00