Commit Graph

256730 Commits

Author SHA1 Message Date
Alex Richardson
90b5fc9583 lib/libc/tests/rpc: Correctly set timeout
The rpc_control() API does not accept the CLCR_SET_RPCB_TIMEOUT command,
it only accepts RPC_SVC_CONNMAXREC_GET/RPC_SVC_CONNMAXREC_SET, so it was
not doing anything.
Instead of incorrectly calling this API, use clnt_create_timed() instead.

I noticed this because the test was timing out after 120s in the CheriBSD CI.

Reviewed By:	ngie
Differential Revision: https://reviews.freebsd.org/D28478
2021-02-13 13:53:01 +00:00
Dimitry Andric
0ff1014944 Update Subversion to 1.14.1 LTS. See contrib/subversion/CHANGES for a
summary of changes, or for a more thorough overview:

https://subversion.apache.org/docs/release-notes/1.14

NOTE 1: There is no need to dump and reload repositories, and the
working copy format is still the same as Subversion 1.8 through 1.13.

NOTE 2: The upstream release also contains a fix for a security issue in
mod_dav_svn (CVE-2020-17525), but since we do not build or use any
Apache modules, it is not an issue for the FreeBSD base system.

Relnotes:	yes
MFC after:	3 days
2021-02-13 14:38:51 +01:00
Kyle Evans
95138d09d2 Regenerate src.conf(5) after FMTREE default change 2021-02-12 23:16:06 -06:00
Kyle Evans
c975494ad7 build: turn off FMTREE by default to prepare for removal
nmtree is derived from fmtree, and has been the default mtree(8) since
6adfbbbf16, a little over a year after its introduction.

fmtree has not seen any substantial work since then, except for build
fixes and runtime issues that were diagnosed in nmtree and backported
because this was still in the tree.

Turn it off by default.

Reviewed by:	bdrewery, brooks, cy, emaste
Differential Revision:	https://reviews.freebsd.org/D28573
2021-02-12 23:16:06 -06:00
Kyle Evans
6fea22cebe fmtree: add a deprecation notice to the manpage
Note that this mtree(8) is actually installed as fmtree(8), while
mtree(8) is located in ^/contrib/mtree -- thus, the reference to
mtree(8) makes a lot more sense in the context in which folks would
actually notice it. Shout-out to Ravi for pointing out that this may
not be an obvious fact.

MFC after:	3 days
Reviewed by:	bdrewery, brooks, cy, emaste
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D28573
2021-02-12 23:15:58 -06:00
Alan Somers
71befc3506 fusefs: set d_off during VOP_READDIR
This allows d_off to be used with lseek to position the file so that
getdirentries(2) will return the next entry.  It is not used by
readdir(3).

PR:		253411
Reported by:	John Millikin <jmillikin@gmail.com>
Reviewed by:	cem
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D28605
2021-02-12 21:50:52 -07:00
Alexander V. Chernikov
a375ec52a7 Fix ifa refcount leak during route addition.
Reported by:	rstone
Reviewed by:	rstone
MFC after:	1 day
2021-02-13 00:06:14 +00:00
Daniel Ebdrup Jensen
5ae8b01832 git-arc(1): Fix nits pointed out in final review
Pointy hat to:	me
2021-02-12 23:39:00 +01:00
Daniel Ebdrup Jensen
30f78a063e git-arc(1): Add manual page
Add manual page based on the usage in the script with a few changes and
hook it up to the build.

Reviewed by:	0mp, markj
Differential Revision:	https://reviews.freebsd.org/D28519
2021-02-12 23:35:02 +01:00
Mark Johnston
62374dfa0f git-arc: Use a separate message file
Rather than putting revision metadata in .git/arc/create-message, create
a tmpfile and use that.  Otherwise arc diff always prompts about it and
in some cases complains because its standard input is piped.

Reported by:	imp
Differential Revision:	https://reviews.freebsd.org/D28614
2021-02-12 15:58:17 -05:00
Alexander V. Chernikov
8ca99aecf7 Fix various NOINET* builds broken by 145bf6c0af.
Reported by:	mjg, bdragon
2021-02-12 20:36:20 +00:00
Alexander V. Chernikov
8170a7d438 Fix interface route addition with net/bird.
The case of adding interface route by specifying interface
 address as the gateway was missed during code refactoring.
Re-add it back by copying non-AF_LINK gateway data when RTF_GATEWAY
 is not set.

Reviewed by:	donner
MFC after:	3 days
2021-02-12 19:45:35 +00:00
Kyle Evans
1253835121 inetd: fix unix sockaddr's length assignment
unsz was always exactly '1' here due to an unfortunate mispositioning
of closing parenthesis. While it's generally irrelevant because bind(2)
is passed the (accurate) sep->se_ctrladdr_size instead, it's not very
helpful for anything locally that wants to use it rather than assuming
that sep->se_ctrladdr_size perfectly fits the end of sun_path.

Just drop unsz entirely and use the result of SUN_LEN() for it.

MFC-after:	3 days
2021-02-12 13:36:38 -06:00
Alan Somers
f540cb27a2 mount_nullfs: rename a local variable
The "source" variable was introduced in r26072, probably as the
traditional counterpart to "target".  But the "source"/"target" names
suggest the opposite of their actual meaning.  With ln, for example, the
source is the real file and the target is the newly created link.  In
mount_nullfs the meaning is the opposite: the target is the existing
file system and the source is the newly created mountpoint.  Better to
use "target"/"mountpoint" terminology, which matches the man page.

MFC after:	6 weeks
Sponsored by:	Axcient
2021-02-12 11:30:52 -07:00
Ed Maste
a62dc346f6 ssh: remove ssh-hpn leftovers
This was introduced in 8998619212, and left behind when the hpn-ssh
patches were removed in 60c59fad88.  Although Being able to log
SO_RCVBUF in debug mode might have some small value on its own, it's
not worth carrying an extra diff against upstream.

Reviewed by:	kevans
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D28610
2021-02-12 10:11:06 -05:00
Richard Scheffenegger
a78fee8182 Adding PRR sysctls to tcp(4) man page
Summary:
Documenting the newly added, and enabled by default,
Proportional Rate Reduction algorithm's governing sysctls.

MFC: 3 days

Reviewed By: kbowling, rgrimes

Differential Revision: https://reviews.freebsd.org/D28568
2021-02-12 12:33:27 +01:00
Kirk McKusick
8563de2f27 Fix bug 253158 - Panic: snapacct_ufs2: bad block - mksnap_ffs(8) crash
The panic reported in 253158 arises because the /mnt/.snap/.factory
snapshot allocated the last block in the filesystem. The snapshot
code allocates the last block in the filesystem as a way of setting
its length to be the size of the filesystem. Part of taking a
snapshot is to remove all the earlier snapshots from the image of
the newest snapshot so that newer snapshots will not claim the blocks
of the earlier snapshots. The panic occurs when the new snapshot
finds that both it and an earlier snapshot claim the same block.

The fix is to set the size of the snapshot to be one block after
the last block in the filesystem. This block can never be allocated
since it is not a valid block in the filesystem. This extra block
is used as a place to store the initial list of blocks that the
snapshot has already copied and is used to avoid a deadlock in and
speed up the ffs_copyonwrite() function.

Reported by:  Harald Schmalzbauer
Tested by:    Peter Holm
PR:           253158
Sponsored by: Netflix
2021-02-11 21:31:16 -08:00
Ed Maste
9e14b918f9 ssh: remove unused variable
This was introduced in 03f6c5cd93, which added use of
sysctl net.inet.ip.portrange.reservedhigh instead of IPPORT_RESERVED,
but it appears the rest of that change was lost in some subsequent
update.

The change should probably be restored, but until then there is no
reason to leave an unused variable around.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2021-02-11 22:22:30 -05:00
Ed Maste
154adbbeb8 ssh: diff reduction against OpenBSD, remove unused includes
These appear to be leftovers from ca86bcf253 and f7167e0ea0

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2021-02-11 21:37:31 -05:00
Konstantin Belousov
adf28ab456 fifo: minor comment and assert improvements.
In particular, replace a note that reload through vget() is obsoleted,
with explanation why this code is required.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:22 +02:00
Konstantin Belousov
26af9f72f7 ffs_unlock: assert that IN_ENDOFF is not leaked past locked scope
This catches both missed processing of IN_ENDOFF and missed application
of VOP_VPUT_PAIR() after VOP that created an entry in the directory.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:22 +02:00
Konstantin Belousov
28703d2713 ffs softdep: Force processing of VI_OWEINACT vnodes when there is inode shortage
Such vnodes prevent inode reuse, and should be force-cleared when ffs_valloc()
is unable to find a free inode.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:22 +02:00
Konstantin Belousov
2011b44fa3 softdep_request_cleanup: wait for softdep_request_clean_flush() to pass
if we noted a parallel request is active and declined to overflow the
system with parallel redundant sync of the vnodes.  But we need to wait
for the flush to finish to see if there are any freed resources.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:22 +02:00
Konstantin Belousov
013168db8c ufs_inactive(): stop hiding ERELOOKUP from ffs_truncate(), return it.
VFS should retry inactivation when possible, then. This should provide
timely removal of unlinked unreferenced inodes.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
b59a8e63d6 Stop ignoring ERELOOKUP from VOP_INACTIVE()
When possible, relock the vnode and retry inactivation.  Only vunref() is
required not to drop the vnode lock, so handle it specially by not retrying.

This is a part of the efforts to ensure that unlinked not referenced vnode
does not prevent inode from reusing.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
6aed2435c8 ufs vnops: brace softdep_prelink() with DOINGSUJ instead of DOINGSOFTDEP
because softdep_prelink() is reverted to NOP for non-J case.  There is no
need to do anything before ufs_direnter() in SU/non-J case, everything
required to sync the directory is done in VOP_VPUT_PAIR().

Suggested by:	mckusick
Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 week
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
ede40b0675 ffs softdep: remove will_direnter argument of softdep_prelink()
Originally this was done in 8a1509e442 to forcibly cover cases
where a hole in the directory could be created by extending into
indirect block, since dependency of writing out indirect block is not
tracked.  This results in excessive amount of fsyncing the directories,
where all creation of new entry forced fsync before it.  This is not needed,
it is enough to fsync when IN_NEEDSYNC is set, and VOP_VPUT_PAIR() provides
the required hook to only perform required syncing.

The series of changes culminating in this commit puts the performance of
metadata-intensive loads back to that before 8a1509e442.

Analyzed by:	mckusick
Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
06f2918ab8 ufs_direnter: directory truncation does not need special case for rename
In ufs_rename case, tdvp is locked from the place where ufs_direnter()
is done till VOP_VPUT_PAIR(), which means that we no longer need to specially
handle rename in ufs_direnter().  Truncation, if possible, is done in the
same way in ffs_vput_pair() both for rename and other VOPs calling
ufs_direnter().  Remove isrename argument and set IN_ENDOFF if
ufs_direnter() succeeded and directory needs truncation.

In ffs_vput_pair(), stop verifying the condition that directory needs
truncation when IN_ENDOFF is set, instead assert that the condition is
true.

Suggested by:	mckusick
Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
038fe6e089 ufs_rename: use VOP_VPUT_PAIR and rely on directory sync/truncation there
Suggested by:	mckusick
Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
74a3652f83 ufs_direnter: move directory truncation to ffs_vput_pair().
VOP_VPUT_PAIR() provides the hook to do the truncation right before
unlock, which is required since truncation might need to fsync(), which
itself might unlock the directory vnode.

Set new flag IN_ENDOFF which indicates that i_endoff is valid and should
be checked against inode size. Excessive size is chomped, but this
operation is advisory and failure to truncate should not result in the
failure of the main VOP.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
30bfb2fa0f ffs_vput_pair(): try harder to recover from the vnode reclaim
In particular, if unlock_vp is false, save vp's inode number and
generation. If ffs_inotovp() can re-create the vnode with the same
number and generation after we finished with handling dvp, then we most
likely raced with unmount, and were able to restore atomicity of open.
We use FFSV_REPLACE_DOOMED there, to drop the old vnode.

This additional recovery is not strictly required, but it improves the
quality of the implementation.

Suggested by:	mckusick
Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
f2c9d038bd FFS: implement special VOP_VPUT_PAIR().
It cleans IN_NEEDSYNC flag on dvp before returning, by applying
ffs_syncvnode() until success or an error different from ERELOOKUP.
IN_NEEDSYNC cleanup is required to avoid creating holes in the directories
when extended into indirect block.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
4a21bcb241 nfsserver: use VOP_VPUT_PAIR().
Apply VOP_VPUT_PAIR() to the end of vnode operations after the
VOP_MKNOD(), VOP_MKDIR(), VOP_LINK(), VOP_SYMLINK(), VOP_CREATE().

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:21 +02:00
Konstantin Belousov
be44e98637 ffs_snapshot: use VOP_VPUT_PAIR after VOP_CREATE.
If the snapshot embrio was reclaimed under us, return error outright.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
3b2aa36024 Use VOP_VPUT_PAIR() for eligible VFS syscalls.
The current list is limited to the cases where UFS needs to handle
vput(dvp) specially. Which means VOP_CREATE(), VOP_MKDIR(), VOP_MKNOD(),
VOP_LINK(), and VOP_SYMLINK().

Reviewed by:	chs, mkcusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
e4aaf35ab5 nullfs: provide special bypass for VOP_VPUT_PAIR
Generic bypass cannot understand the rules of liveness for the VOP.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
49c117c193 Add VOP_VPUT_PAIR() with trivial default implementation.
The VOP is intended to be used in situations where VFS has two
referenced locked vnodes, typically a directory vnode dvp and a vnode
vp that is linked from the directory, and at least dvp is vput(9)ed.
The child vnode can be also vput-ed, but optionally left referenced and
locked.

There, at least UFS may need to do some actions with dvp which cannot be
done while vp is also locked, so its lock might be dropped temporary.
For instance, in some cases UFS needs to sync dvp to avoid filesystem
state that is currently not handled by either kernel nor fsck. Having
such VOP provides the neccessary context for filesystem which can do
correct locking and handle potential reclamation of vp after relock.

Trivial implementation does vput(dvp) and optionally vput(vp).

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
ee965dfa64 vn_open(): If the vnode is reclaimed during open(2), do not return error.
Most future operations on the returned file descriptor will fail
anyway, and application should be ready to handle that failures.  Not
forcing it to understand the transient failure mode on open, which is
implementation-specific, should make us less special without loss of
reporting of errors.

Suggested by: chs
Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
08c2dc2841 ufs_direnter/SU: unconditionally UFS_UPDATE inode when extending directory
for all kinds of async/SU mount variants.

Submitted by:	mckusick
Reviewed by:	chs
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
1de1e2bfbf ffs_syncvnode: only clear IN_NEEDSYNC after successfull sync
If it is cleaned before the sync, other threads might see the inode without
the flag set, because syncing could unlock it.

Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
89fd61d955 Merge ufs_fhtovp() into ffs_inotovp().
The function alone was not used for anything but ffs_fstovp() for long time.

Suggested by:	mckusick
Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
5952c86c78 ffs_inotovp(): interface to convert (ino, gen) into alive vnode
It generalizes the VFS_FHTOVP() interface, making it possible to fetch
the inode without faking filehandle.  Also it adds the ffs flags argument
which allows to control ffs_vgetf() call.

Requested by:	mckusick
Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
f16c26b1c0 ffs: Add FFSV_REPLACE_DOOMED flag to ffs_vgetf()
It specifies that caller requests a fresh non-doomed vnode.  If doomed
vnode is found in the hash, it should behave similarly to FFSV_REPLACE.
Or, to put it differently, the flag is same as FFSV_REPLACE, but only
when the found hashed vnode is doomed.

Reviewed by:	chs, mkcusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:20 +02:00
Konstantin Belousov
e94f2f1be3 ffs: call ufsdirhash_dirtrunc() right after setting directory size
Later processing of ffs_truncate() might temporary unlock the directory
vnode, causing unsychronized dirhash and inode sizes if update is
postponed to UFS_TRUNCATE() callers.

Reviewed by:	chs, mkcusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2021-02-12 03:02:19 +02:00
Konstantin Belousov
bf0db19339 buf SU hooks: track buf_start() calls with B_IOSTARTED flag
and only call buf_complete() if previously started.  Some error paths,
like CoW failire, might skip buf_start() and do bufdone(), which itself
call buf_complete().

Various SU handle_written_XXX() functions check that io was started
and incomplete parts of the buffer data reverted before restoring them.
This is a useful invariant that B_IO_STARTED on buffer layer allows to
keep instead of changing check and panic into check and return.

Reported by:	pho
Reviewed by:	chs, mckusick
Tested by:	pho
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundations
2021-02-12 03:02:19 +02:00
Konstantin Belousov
0281f88e5d ffs_vnops.c: Move opt_*.h includes to the top.
as it is done in other places.  Header files might need options defined
for correct operation.

Reviewed by:	chs, mckusick
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2021-02-12 03:02:19 +02:00
Kyle Evans
b2c4ca8d28 pkg(7): address minor nits (mostly clang-analyze complaints)
- One (1) spurious whitespace.
- One (1) occurrence of "random(3) bad, arc4random(3)" good.
- Three (3) writes that will never be seen.

The latter two points are complaints from clang-analyze. Switching to
arc4random(3) is decidedly a good idea because we weren't doing any kind
of PRNG seeding anyways. The discarded assignments are arguably good
for future-proofing, but it's better to improve the S/N ratio from
clang-analyze.

Reviewed by:	bapt, manu
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D28525
2021-02-11 18:58:27 -06:00
Kyle Evans
18418e1936 pkg(7): add an -r reponame option for bootstrap and add
This is limited to bootstrap/add because some real pkg(8) commands
have -r flags with an incompatible meaning/usage, e.g., pkg-audit.
pkg(7) will still commence the search as it has, but it will ignore any
repo objects without the given name so that overrides and whatnot still
work as expected.

The use of it for add is noted in the manpage; notably, that the
signature config for that repository will be used over global config if
it's specified. i.e., pkg(7) should assume that the given pkg did come
from that repository and treat it appropriately.

Reviewed by:	bapt, manu
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D28524
2021-02-11 18:58:26 -06:00
Jung-uk Kim
bf80e08ed5 yacc: Regen test cases for the previous commit 2021-02-11 18:41:28 -05:00
Jung-uk Kim
80f3143274 yacc: Use NULL instead of 0 for pointers
Note araujo tried to fix it in r298241 but he only touched generated
files for bootstrap.  This commit properly fixes the problem.
2021-02-11 18:40:00 -05:00