139 Commits

Author SHA1 Message Date
Alan Somers
102c7ac083 fusefs: handle ENOSYS for FUSE_INTERRUPT
Though it's not documented, Linux will interpret a FUSE_INTERRUPT response
of ENOSYS as "the file system does not support FUSE_INTERRUPT".
Subsequently it will never send FUSE_INTERRUPT again to the same mount
point.  This change matches Linux's behavior.

PR:		346357
Sponsored by:	The FreeBSD Foundation
2019-04-24 17:30:50 +00:00
Alan Somers
ebbfe00ec2 fusefs: interruptibility improvements suggested by kib
* Block stop signals in fticket_wait_answer
* Hold ps_mtx while checking signal disposition
* style(9) changes

PR:		346357
Reported by:	kib
Sponsored by:	The FreeBSD Foundation
2019-04-24 15:54:18 +00:00
Alan Somers
21d4686c5c fusefs: diff reduction between fuse_read_biobackend and ext_read
The main difference is to replace some custom logic with bread.  No
functional change at this point, but this is one step towards adding
readahead.

Sponsored by:	The FreeBSD Foundation
2019-04-23 22:34:32 +00:00
Alan Somers
bad3de4365 fusefs: use vfs_bio_clrbuf in fuse_vnode_setsize
Reuse fuse_vnode_setsize instead of reinventing the wheel.  This is what
ext2_ind_truncate does.

PR:		233783
Sponsored by:	The FreeBSD Foundation
2019-04-23 22:25:50 +00:00
Alan Somers
419e7ff674 fusefs: rename the SDT probes from "fuse" to "fusefs"
This matches the new name of the kld.

Sponsored by:	The FreeBSD Foundation
2019-04-20 00:04:31 +00:00
Alan Somers
4423ae76ca fusefs: reap dead code
Sponsored by:	The FreeBSD Foundation
2019-04-19 23:04:07 +00:00
Alan Somers
268c28edbc fusefs: give priority to FUSE_INTERRUPT operations
When interrupting a FUSE operation, send the FUSE_INTERRUPT op to the daemon
ASAP, ahead of other unrelated operations.

PR:		236530
Sponsored by:	The FreeBSD Foundation
2019-04-19 21:50:23 +00:00
Alan Somers
f0f7fc1be4 fusefs: fix interrupting FUSE_SETXATTR
fusefs's VOP_SETEXTATTR calls uiomove(9) before blocking, so it can't be
restarted.  It must be interrupted instead.

PR:		236530
Sponsored by:	The FreeBSD Foundation
2019-04-19 20:31:12 +00:00
Alan Somers
3d070fdc76 fusefs: don't send FUSE_INTERRUPT for ops that are still in-kernel
If a pending FUSE operation hasn't yet been sent to the daemon, then there's
no reason to inform the daemon that it's been interrupted.  Instead, simply
remove it from the fuse message queue and set its status to EINTR or
ERESTART as appropriate.

PR:		346357
Sponsored by:	The FreeBSD Foundation
2019-04-19 15:05:32 +00:00
Alan Somers
a154214620 fusefs: improvements to interruptibility
* If a process receives a fatal signal while blocked on a fuse operation,
  return ASAP without waiting for the operation to complete.  But still send
  the FUSE_INTERRUPT op to the daemon.
* Plug memory leaks from r346339

Interruptibility is now fully functional, but it could be better:
* Operations that haven't been sent to the server yet should be aborted
  without sending FUSE_INTERRUPT.
* It would be great if write operations could be made restartable.
  That would require delaying uiomove until the last possible moment, which
  would be sometime during fuse_device_read.
* It would be nice if we didn't have to guess which EAGAIN responses were
  for FUSE_INTERRUPT operations.

PR:		236530
Sponsored by:	The FreeBSD Foundation
2019-04-18 19:16:34 +00:00
Alan Somers
723c776829 fusefs: WIP making FUSE operations interruptible
The fuse protocol includes a FUSE_INTERRUPT operation that the client can
send to the server to indicate that it wants to abort an in-progress
operation.  It's required to interrupt any syscall that is blocking on a
fuse operation.

This commit adds basic FUSE_INTERRUPT support.  If a process receives any
signal while it's blocking on a FUSE operation, it will send a
FUSE_INTERRUPT and wait for the original operation to complete.  But there
is still much to do:

* The current code will leak memory if the server ignores FUSE_INTERRUPT,
  which many do.  It will also leak memory if the server completes the
  original operation before it receives the FUSE_INTERRUPT.
* An interrupted read(2) will incorrectly appear to be successful.
* fusefs should return immediately for fatal signals.
* Operations that haven't been sent to the server yet should be aborted
  without sending FUSE_INTERRUPT.
* Test coverage should be better.
* It would be great if write operations could be made restartable.
  That would require delaying uiomove until the last possible moment, which
  would be sometime during fuse_device_read.

PR:		236530
Sponsored by:	The FreeBSD Foundation
2019-04-17 23:32:38 +00:00
Alan Somers
f067b60946 fusefs: implement VOP_ADVLOCK
PR:		234581
Sponsored by:	The FreeBSD Foundation
2019-04-12 23:22:27 +00:00
Alan Somers
6af6fdcea7 fusefs: evict invalidated cache contents during write-through
fusefs's default cache mode is "writethrough", although it currently works
more like "write-around"; writes bypass the cache completely.  Since writes
bypass the cache, they were leaving stale previously-read data in the cache.
This commit invalidates that stale data.  It also adds a new global
v_inval_buf_range method, like vtruncbuf but for a range of a file.

PR:		235774
Reported by:	cem
Sponsored by:	The FreeBSD Foundation
2019-04-12 19:05:06 +00:00
Alan Somers
1f4a83f981 fusefs: Handle ENOSYS for all remaining opcodes
For many FUSE opcodes, an error of ENOSYS has special meaning.  fusefs
already handled some of those; this commit adds handling for the remainder:

* FUSE_FSYNC, FUSE_FSYNCDIR: ENOSYS means "success, and automatically return
  success without calling the daemon from now on"
* All extattr operations: ENOSYS means "fail EOPNOTSUPP, and automatically
  do it without calling the daemon from now on"

PR:		236557
Sponsored by:	The FreeBSD Foundation
2019-04-12 00:15:36 +00:00
Alan Somers
b4227f34e8 fusefs: /dev/fuse should be 0666
The fuse protocol is designed with security in mind.  It prevents users from
spying on each others' activities.  And it doesn't grant users any
privileges that they didn't already have.  So it's appropriate to make it
available to everyone.  Plus, it's necessary in order for kyua to run tests
as an unprivileged user.

Sponsored by:	The FreeBSD Foundation
2019-04-11 22:43:19 +00:00
Alan Somers
c9c34c2057 fusefs: test that we reparent a vnode during rename
fusefs tracks each vnode's parent.  The rename code was already correctly
updating it.  Delete a comment that said otherwise, and add a regression
test for it.

Sponsored by:	The FreeBSD Foundation
2019-04-11 22:34:28 +00:00
Alan Somers
64f31d4f3b fusefs: fix a panic in a stale vnode situation
Don't panic if the server changes the file type of a file without us first
deleting it.  That could indicate a buggy server, but it could also be the
result of one of several race conditions.  Return EAGAIN as we do elsewhere.

Sponsored by:	The FreeBSD Foundation
2019-04-11 22:32:34 +00:00
Alan Somers
4683b90591 fusefs: don't disappear a vnode on entry cache expiration
When the entry cache expires, it's only necessary to purge the cache.
Disappearing a vnode also purges the attribute cache, which is unnecessary,
and invalidates the data cache, which could be harmful.

Sponsored by:	The FreeBSD Foundation
2019-04-11 21:13:54 +00:00
Alan Somers
6124fd7106 fusefs: Finish supporting -o default_permissions
I got most of -o default_permissions working in r346088.  This commit adds
sticky bit checks.  One downside is that sometimes there will be an extra
FUSE_GETATTR call for the parent directory during unlink or rename.  But in
actual use I think those attributes will almost always be cached.

PR:		216391
Sponsored by:	The FreeBSD Foundation
2019-04-11 21:00:40 +00:00
Alan Somers
dc14d593a6 fusefs: use vn_vget_ino_gen in fuse_vnop_lookup
vn_vget_ino_gen is a helper function added in r268606 to simplify cases just
like this.

Sponsored by:	The FreeBSD Foundation
2019-04-11 17:20:15 +00:00
Alan Somers
438b8a6fa2 fusefs: eliminate a superfluous FUSE_GETATTR from VOP_LOOKUP
fuse_vnop_lookup was using a FUSE_GETATTR operation when looking up "." and
"..", even though the only information it needed was the file type and file
size.  "." and ".." are obviously always going to be directories; there's no
need to double check.

Sponsored by:	The FreeBSD Foundation
2019-04-11 05:11:02 +00:00
Alan Somers
73825da397 fusefs: remove "early permission check hack"
fuse_vnop_lookup contained an awkward hack meant to reduce daemon activity
during long lookup chains.  However, the hack is no longer necessary now
that we properly cache file attributes.  Also, I'm 99% certain that it
could've bypassed permission checks when using openat to open a file
relative to a directory that lacks execute permission.

Sponsored by:	The FreeBSD Foundation
2019-04-10 21:46:59 +00:00
Alan Somers
666f8543bb fusefs: various cleanups
* Eliminate fuse_access_param.  Whatever it was supposed to do, it seems
  like it was never complete.  The only real function it ever seems to have
  had was a minor performance optimization, which I've already eliminated.
* Make extended attribute operations obey the allow_other mount option.
* Allow unprivileged access to the SYSTEM extattr namespace when
  -o default_permissions is not in use.
* Disallow setextattr and deleteextattr on read-only mounts.
* Add tests for a few more error cases.

Sponsored by:	The FreeBSD Foundation
2019-04-10 21:10:21 +00:00
Alan Somers
ff4fbdf548 fusefs: WIP supporting -o default_permissions
Normally all permission checking is done in the fuse server.  But when -o
default_permissions is used, it should be done in the kernel instead.  This
commit adds appropriate permission checks through fusefs when -o
default_permissions is used.  However, sticky bit checks aren't working yet.
I'll handle those in a follow-up commit.

There are no checks for file flags, because those aren't supported by our
version of the FUSE protocol.  Nor is there any support for ACLs, though
that could be added if there were any demand.

PR:		216391
Reported by:	hiyorin@gmail.com
Sponsored by:	The FreeBSD Foundation
2019-04-10 17:31:00 +00:00
Alan Somers
44f10c6e40 fusefs: cache negative lookups
The FUSE protocol includes a way for a server to tell the client that a
negative lookup response is cacheable for a certain amount of time.

PR:		236226
Sponsored by:	The FreeBSD Foundation
2019-04-09 21:22:02 +00:00
Alan Somers
ccb75e4939 fusefs: implement entry cache timeouts
Follow-up to r346046.  These two commits implement fuse cache timeouts for
both entries and attributes.  They also remove the vfs.fusefs.lookup_cache
enable sysctl, which is no longer needed now that cache timeouts are
honored.

PR:		235773
Sponsored by:	The FreeBSD Foundation
2019-04-09 17:23:34 +00:00
Alan Somers
3f2c630c74 fusefs: implement attribute cache timeouts
The FUSE protocol allows the server to specify the timeout period for the
client's attribute and entry caches.  This commit implements the timeout
period for the attribute cache.  The entry cache's timeout period is
currently disabled because it panics, and is guarded by the
vfs.fusefs.lookup_cache_expire sysctl.

PR:		235773
Reported by:	cem
Sponsored by:	The FreeBSD Foundation
2019-04-09 00:47:38 +00:00
Alan Somers
cad677915f fusefs: cache file attributes
FUSE_LOOKUP, FUSE_GETATTR, FUSE_SETATTR, FUSE_MKDIR, FUSE_LINK,
FUSE_SYMLINK, FUSE_MKNOD, and FUSE_CREATE all return file attributes with a
cache validity period.  fusefs will now cache the attributes, if the server
returns a non-zero cache validity period.

This change does _not_ implement finite attr cache timeouts.  That will
follow as part of PR 235773.

PR:		235775
Reported by:	cem
Sponsored by:	The FreeBSD Foundation
2019-04-08 18:45:41 +00:00
Alan Somers
caf5f57d2d fusefs: implement VOP_ACCESS
VOP_ACCESS was never fully implemented in fusefs.  This change:
* Removes the FACCESS_DO_ACCESS flag, which pretty much disabled the whole
  vop.
* Removes a quixotic special case for VEXEC on regular files.  I don't know
  why that was in there.
* Removes another confusing special case for VADMIN.
* Removes the FACCESS_NOCHECKSPY flag.  It seemed to be a performance
  optimization, but I'm unconvinced that it was a net positive.
* Updates test cases.

This change does NOT implement -o default_permissions.  That will be handled
separately.

PR:		236291
Sponsored by:	The FreeBSD Foundation
2019-04-05 18:37:48 +00:00
Alan Somers
efa23d9784 fusefs: enforce -onoallow_other even beneath the mountpoint
When -o allow_other is not in use, fusefs is supposed to prevent access to
the filesystem by any user other than the one who owns the daemon.  Our
fusefs implementation was only enforcing that restriction at the mountpoint
itself.  That was usually good enough because lookup usually descends from
the mountpoint.  However, there are cases when it doesn't, such as when
using openat relative to a file beneath the mountpoint.

PR:		237052
Sponsored by:	The FreeBSD Foundation
2019-04-05 17:21:23 +00:00
Alan Somers
140bb4927a fusefs: correctly return EROFS from VOP_ACCESS
Sponsored by:	The FreeBSD Foundation
2019-04-05 15:33:43 +00:00
Alan Somers
a7e81cb3db fusefs: properly handle FOPEN_KEEP_CACHE
If a fuse file system returne FOPEN_KEEP_CACHE in the open or create
response, then the client is supposed to _not_ clear its caches for that
file.  I don't know why clearing the caches would be the default given that
there's a separate flag to bypass the cache altogether, but that's the way
it is.  fusefs(5) will now honor this flag.

Our behavior is slightly different than Linux's because we reuse file
handles.  That means that open(2) wont't clear the cache if there's a
reusable file handle, even if the file server wouldn't have sent
FOPEN_KEEP_CACHE had we opened a new file handle like Linux does.

PR:		236560
Sponsored by:	The FreeBSD Foundation
2019-04-04 20:30:14 +00:00
Alan Somers
8d013bec7a fusefs: fix some uninitialized memory references
This bug was long present, but was exacerbated by r345876.

The problem is that fiov_refresh was bzero()ing a buffer _before_ it
reallocated that buffer.  That's obviously the wrong order.  I fixed the
order in r345876, which exposed the main problem.  Previously, the first 160
bytes of the buffer were getting bzero()ed when it was first allocated in
fiov_init.  Subsequently, as that buffer got recycled between callers, the
portion used by the _previous_ caller was getting bzero()ed by the current
caller in fiov_refresh.  The problem was never visible simply because no
caller was trying to use more than 160 bytes.

Now the buffer gets properly bzero()ed both at initialization time and any
time it gets enlarged or reallocated.

Sponsored by:	The FreeBSD Foundation
2019-04-04 20:24:58 +00:00
Alan Somers
9a696dc6bb MFHead@r345880 2019-04-04 18:26:32 +00:00
Alan Somers
12292a99ac fusefs: correctly handle short writes
If a FUSE daemon returns FOPEN_DIRECT_IO when a file is opened, then it's
allowed to write less data than was requested during a FUSE_WRITE operation
on that file handle.  fusefs should simply return a short write to userland.

The old code attempted to resend the unsent data.  Not only was that
incorrect behavior, but it did it in an ineffective way, by attempting to
"rewind" the uio and uiomove the unsent data again.

This commit correctly handles short writes by returning directly to
userland if FOPEN_DIRECT_IO was set.  If it wasn't set (making the short
write technically a protocol violation), then we resend the unsent data.
But instead of rewinding the uio, just resend the data that's already in the
kernel.

That necessitated a few changes to fuse_ipc.c to reduce the amount of bzero
activity.  fusefs may be marginally faster as a result.

PR:		236381
Sponsored by:	The FreeBSD Foundation
2019-04-04 16:51:34 +00:00
Alan Somers
35cf0e7e56 fusefs: fix a panic in VOP_READDIR
The original fusefs import, r238402, contained a bug in fuse_vnop_close that
could close a directory's file handle while there were still other open file
descriptors.  The code looks deliberate, but there is no explanation for it.
This necessitated a workaround in fuse_vnop_readdir that would open a new
file handle if, "for some mysterious reason", that vnode didn't have any
open file handles.  r345781 had the effect of causing the workaround to
panic, making the problem more visible.

This commit removes the workaround and the original bug, which also fixes
the panic.

Sponsored by:	The FreeBSD Foundation
2019-04-03 20:57:43 +00:00
Alan Somers
9f10f423a9 fusefs: send FUSE_FLUSH during VOP_CLOSE
The FUSE protocol says that FUSE_FLUSH should be send every time a file
descriptor is closed.  That's not quite possible in FreeBSD because multiple
file descriptors can share a single struct file, and closef doesn't call
fo_close until the last close.  However, we can still send FUSE_FLUSH on
every VOP_CLOSE, which is probably good enough.

There are two purposes for FUSE_FLUSH.  One is to allow file systems to
return EIO if they have an error when writing data that's cached
server-side.  The other is to release POSIX file locks (which fusefs(5) does
not yet support).

PR:		236405, 236327
Sponsored by:	The FreeBSD Foundation
2019-04-03 19:59:45 +00:00
Alan Somers
e312493b37 fusefs: during ftruncate, discard cached data past truncation point
During truncate, fusefs was discarding entire cached blocks, but it wasn't
zeroing out the unused portion of a final partial block.  This resulted in
reads returning stale data.

PR:		233783
Reported by:	fsx
Sponsored by:	The FreeBSD Foundation
2019-04-03 02:29:56 +00:00
Alan Somers
d3a8f2dd09 fusefs: fix a just-introduced panic in readdir
r345808 changed the interface of fuse_filehandle_open, but failed to update
one caller.

Sponsored by:	The FreeBSD Foundation
2019-04-02 19:20:55 +00:00
Alan Somers
9e4448719b fusefs: cleanup and refactor some recent commits
This commit cleans up after recent commits, especially 345766, 345768, and
345781.  There is no functional change.  The most important change is to add
comments documenting why we can't send flags like O_APPEND in
FUSE_WRITE_OPEN.

PR:		236340
Sponsored by:	The FreeBSD Foundation
2019-04-02 18:09:40 +00:00
Alan Somers
f8d4af104b fusefs: send FUSE_OPEN for every open(2) with unique credentials
By default, FUSE performs authorization in the server.  That means that it's
insecure for the client to reuse FUSE file handles between different users,
groups, or processes.  Linux handles this problem by creating a different
FUSE file handle for every file descriptor.  FreeBSD can't, due to
differences in our VFS design.

This commit adds credential information to each fuse_filehandle.  During
open(2), fusefs will now only reuse a file handle if it matches the exact
same access mode, pid, uid, and gid of the calling process.

PR:		236844
Sponsored by:	The FreeBSD Foundation
2019-04-01 20:42:15 +00:00
Alan Somers
363a74163b fusefs: allow opening files O_EXEC
O_EXEC is useful for fexecve(2) and fchdir(2).  Treat it as another fufh
type alongside the existing RDONLY, WRONLY, and RDWR.  Prior to r345742 this
would've caused a memory and performance penalty.

PR:		236329
Sponsored by:	The FreeBSD Foundation
2019-04-01 16:36:02 +00:00
Alan Somers
4a6d5507f7 fusefs: fix an inverted error check in my last commit
This should be merged alongside 345766

Sponsored by:	The FreeBSD Foundation
2019-04-01 16:15:29 +00:00
Alan Somers
5ec10aa527 fusefs: replace obsolete array idioms
r345742 replaced fusefs's fufh array with a fufh list.  But it left a few
array idioms in place.  This commit replaces those idioms with more
efficient list idioms.  One location is in fuse_filehandle_close, which now
takes a pointer argument.  Three other locations are places that had to loop
over all of a vnode's fuse filehandles.

Sponsored by:	The FreeBSD Foundation
2019-04-01 14:23:43 +00:00
Alan Somers
1cedd6dfac fusefs: replace the fufh table with a linked list
The FUSE protocol allows each open file descriptor to have a unique file
handle.  On FreeBSD, these file handles must all be stored in the vnode.
The old method (also used by OSX and OpenBSD) is to store them all in a
small array.  But that limits the total number that can be stored.  This
commit replaces the array with a linked list (a technique also used by
Illumos).  There is not yet any change in functionality, but this is the
first step to fixing several bugs.

PR:		236329, 236340, 236381, 236560, 236844
Discussed with:	cem
Sponsored by:	The FreeBSD Foundation
2019-03-31 03:19:10 +00:00
Alan Somers
5fccbf313a fusefs: don't force direct io for files opened O_WRONLY
Previously fusefs would treat any file opened O_WRONLY as though the
FOPEN_DIRECT_IO flag were set, in an attempt to avoid issuing reads as part
of a RMW write operation on a cached part of the file.  However, the FUSE
protocol explicitly allows reads of write-only files for precisely that
reason.

Sponsored by:	The FreeBSD Foundation
2019-03-30 00:57:07 +00:00
Alan Somers
f220ef0b35 fix the GENERIC-NODEBUG build after r345675
Submitted by:	cy
Reported by:	cy, Michael Butler <imb@protected-networks.net>
MFC after:	2 weeks
X-MFC-With:	345675
2019-03-29 14:07:30 +00:00
Alan Somers
080518d810 fusefs: convert debug printfs into dtrace probes
fuse(4) was heavily instrumented with debug printf statements that could
only be enabled with compile-time flags. They fell into three basic groups:

1. Totally redundant with dtrace FBT probes. These I deleted.
2. Print textual information, usually error messages. These I converted to
   SDT probes of the form fuse:fuse:FILE:trace. They work just like the old
   printf statements except they can be enabled at runtime with dtrace. They
   can be filtered by FILE and/or by priority.
3. More complicated probes that print detailed information. These I
   converted into ad-hoc SDT probes.

Also, de-inline fuse_internal_cache_attrs.  It's big enough to be a regular
function, and this way it gets a dtrace FBT probe.

This commit is a merge of r345304, r344914, r344703, and r344664 from
projects/fuse2.

Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19667
2019-03-29 02:13:06 +00:00
Alan Somers
98852a32af fusefs: fix error handling in fuse_vnop_strategy
Reported by:	cem
Sponsored by:	The FreeBSD Foundation
2019-03-28 21:57:42 +00:00
Alan Somers
f203d1734d fusefs: don't ignore errors in fuse_vnode_refreshsize
Reported by:	Coverity
Coverity CID:	1368622
Sponsored by:	The FreeBSD Foundation
2019-03-27 16:45:30 +00:00