Commit Graph

165 Commits

Author SHA1 Message Date
Alan Somers
6fa772a88e fusefs: skip the Write.mmap test when mmap is not available
fusefs doesn't not allow mmap when data caching is disabled.

Sponsored by:	The FreeBSD Foundation
2019-06-17 17:17:01 +00:00
Alan Somers
d569012f45 fusefs: implement non-clustered readahead
fusefs will now read ahead at most one cache block at a time (usually 64
KB).  Clustered reads are still TODO.  Individual file systems may disable
read ahead by setting fuse_init_out.max_readahead=0 during initialization.

Sponsored by:	The FreeBSD Foundation
2019-06-17 16:56:51 +00:00
Alan Somers
eadd12d35d fusefs: rename the ReadCacheable.default_readahead test
The test didn't actually have anything to do with readahead.  Rename it to
"ReadCacheable.cache_block"

Sponsored by:	The FreeBSD Foundation
2019-06-17 14:42:27 +00:00
Alan Somers
b5aaf286ea fusefs: fix the "write-through" of write-through cacheing
Our fusefs(5) module supports three cache modes: uncached, write-through,
and write-back.  However, the write-through mode (which is the default) has
never actually worked as its name suggests.  Rather, it's always been more
like "write-around".  It wrote directly, bypassing the cache.  The cache
would only be populated by a subsequent read of the same data.

This commit fixes that problem.  Now the write-through mode works as one
would expect: write(2) immediately adds data to the cache and then blocks
while the daemon processes the write operation.

A side effect of this change is that non-cache-block-aligned writes will now
incur a read-modify-write cycle of the cache block.  The old behavior
(bypassing write cache entirely) can still be achieved by opening a file
with O_DIRECT.

PR:		237588
Sponsored by:	The FreeBSD Foundation
2019-06-14 19:47:48 +00:00
Alan Somers
8eecd9ce05 fusefs: enable write clustering
Enable write clustering in fusefs whenever cache mode is set to writeback
and the "async" mount option is used.  With default values for MAXPHYS,
DFLTPHYS, and the fuse max_write mount parameter, that means sequential
writes will now be written 128KB at a time instead of 64KB.

Also, add a regression test for PR 238565, a panic during unmount that
probably affects UFS, ext2, and msdosfs as well as fusefs.

PR:		238565
Sponsored by:	The FreeBSD Foundation
2019-06-14 18:14:51 +00:00
Alan Somers
dff3a6b410 fusefs: fix a bug with WriteBack cacheing
An errant vfs_bio_clrbuf snuck in in r348931.  Surprisingly, it doesn't have
any effect most of the time.  But under some circumstances it cause the
buffer to behave in a write-only fashion.

Sponsored by:	The FreeBSD Foundation
2019-06-13 19:07:03 +00:00
Alan Somers
93c0c1d4ce fusefs: fix a page fault with writeback cacheing
When truncating a file downward through a dirty buffer, it's neccessary to
update the buffer's b->dirtyend.

Sponsored by:	The FreeBSD Foundation
2019-06-11 23:46:31 +00:00
Alan Somers
a87e0831ab fusefs: WIP fixing writeback cacheing
The current "writeback" cache mode, selected by the
vfs.fusefs.data_cache_mode sysctl, doesn't do writeback cacheing at all.  It
merely goes through the motions of using buf(9), but then writes every
buffer synchronously.  This commit:

* Enables delayed writes when the sysctl is set to writeback cacheing
* Fixes a cache-coherency problem when extending a file whose last page has
  just been written.
* Removes the "sync" mount option, which had been set unconditionally.
* Adjusts some SDT probes
* Adds several new tests that mimic what fsx does but with more control and
  without a real file system.  As I discover failures with fsx, I add
  regression tests to this file.
* Adds a test that ensures we can append to a file without reading any data
  from it.

This change is still incomplete.  Clustered writing is not yet supported,
and there are frequent "panic: vm_fault_hold: fault on nofault entry" panics
that I need to fix.

Sponsored by:	The FreeBSD Foundation
2019-06-11 16:32:33 +00:00
Alan Somers
b690d120d9 fusefs: fix an infinite loop in the tests
It was possible for the MockFS thread to infinitely loop if it got an error
reading from /dev/fuse.

Sponsored by:	The FreeBSD Foundation
2019-06-11 16:16:16 +00:00
Alan Somers
d53a96f16f fusefs: fix a comment. No functional change.
Sponsored by:	The FreeBSD Foundation
2019-06-10 22:23:37 +00:00
Alan Somers
2d6bf515df fusefs: add some explicit tests for FUSE_FORGET
Sponsored by:	The FreeBSD Foundation
2019-06-06 16:29:08 +00:00
Alan Somers
d2654efffe fusefs: remove debugging code that accidentally snuck into r348365
Sponsored by:	The FreeBSD Foundation
2019-06-04 19:06:24 +00:00
Alan Somers
a639731ba9 fusefs: respect RLIMIT_FSIZE
Sponsored by:	The FreeBSD Foundation
2019-06-03 23:24:07 +00:00
Alan Somers
6ff7f297f8 fusefs: don't require FUSE_EXPORT_SUPPORT for async invalidation
In r348560 I thought that FUSE_EXPORT_SUPPORT was required for cases where
the node to be invalidated (or the parent of the entry to be invalidated)
wasn't cached.  But I realize now that that's not the case.  During entry
invalidation, if the parent isn't in the vfs hash table, then it must've
been reclaimed.  And since fuse_vnop_reclaim does a cache_purge, that means
the entry to be invalidated has already been removed from the namecache.
And during inode invalidation, if the inode to be invalidated isn't in the
vfs hash table, then it too must've been reclaimed.  In that case it will
have no buffer cache to invalidate.

Sponsored by:	The FreeBSD Foundation
2019-06-03 20:45:32 +00:00
Alan Somers
eae1ae132c fusefs: support asynchronous cache invalidation
Protocol 7.12 adds a way for the server to notify the client that it should
invalidate an inode's data cache and/or attributes.  This commit implements
that mechanism.  Unlike Linux's implementation, ours requires that the file
system also supports FUSE_EXPORT_SUPPORT (NFS-style lookups).  Otherwise the
invalidation operation will return EINVAL.

Sponsored by:	The FreeBSD Foundation
2019-06-03 17:34:01 +00:00
Alan Somers
c2d70d6e6f fusefs: support name cache invalidation
Protocol 7.12 adds a way for the server to notify the client that it should
invalidate an entry from its name cache.  This commit implements that
mechanism.

Sponsored by:	The FreeBSD Foundation
2019-06-01 00:11:19 +00:00
Alan Somers
0d2bf48996 fusefs: check the vnode cache when looking up files for the NFS server
FUSE allows entries to be cached for a limited amount of time.  fusefs's
vnop_lookup method already implements that using the timeout functionality
of cache_lookup/cache_enter_time.  However, lookups for the NFS server go
through a separate path: vfs_vget.  That path can't use the same timeout
functionality because cache_lookup/cache_enter_time only work on pathnames,
whereas vfs_vget works by inode number.

This commit adds entry timeout information to the fuse vnode structure, and
checks it during vfs_vget.  This allows the NFS server to take advantage of
cached entries.  It's also the same path that FUSE's asynchronous cache
invalidation operations will use.

Sponsored by:	The FreeBSD Foundation
2019-05-31 21:22:58 +00:00
Alan Somers
a34cdd26d0 fusefs: prefer FUSE_ROOT_ID to literal 1 in the tests
Sponsored by:	The FreeBSD Foundation
2019-05-31 17:02:37 +00:00
Alan Somers
a4856c96d0 fusefs: raise protocol level to 7.12
This commit raises the protocol level and adds backwards-compatibility code
to handle structure size changes.  It doesn't implement any new features.
The new features added in protocol 7.12 are:

* server-side umask processing (which FreeBSD won't do)
* asynchronous inode and directory entry invalidation (which I'll do next)

Sponsored by:	The FreeBSD Foundation
2019-05-29 16:39:52 +00:00
Alan Somers
d4fd0c8148 fusefs: set the flags fields of fuse_write_in and fuse_read_in
These fields are supposed to contain the file descriptor flags as supplied
to open(2) or set by fcntl(2).  The feature is kindof useless on FreeBSD
since we don't supply all of these flags to fuse (because of the weak
relationship between struct file and struct vnode).  But we should at least
set the access mode flags (O_RDONLY, etc).

This is the last fusefs change needed to get full protocol 7.9 support.
There are still a few options we don't support for good reason (mandatory
file locking is dumb, flock support is broken in the protocol until 7.17,
etc), but there's nothing else to do at this protocol level.

Sponsored by:	The FreeBSD Foundation
2019-05-28 01:09:19 +00:00
Alan Somers
8aa24ed381 fusefs: flock(2) locks must be implemented in-kernel
If a FUSE file system sets the FUSE_POSIX_LOCKS flag then it can support
fcntl(2)-style locks directly.  However, the protocol does not adequately
support flock(2)-style locks until revision 7.17.  They must be implemented
locally in-kernel instead.  This unfortunately breaks the interoperability
of fcntl(2) and flock(2) locks for file systems that support the former.
C'est la vie.

Prior to this commit flock(2) would get sent to the server as a
fcntl(2)-style lock with the lock owner field set to stack garbage.

Sponsored by:	The FreeBSD Foundation
2019-05-28 00:03:46 +00:00
Alan Somers
b7774b82cb fusefs: fix an alignment issue in the tests on arm
Sponsored by:   The FreeBSD Foundation
2019-05-27 21:51:43 +00:00
Alan Somers
bda39894c5 fusefs: set FUSE_WRITE_CACHE when writing from cache
This bit tells the server that we're not sure which uid, gid, and/or pid
originated the write.  I don't know of a single file system that cares, but
it's part of the protocol.

Sponsored by:	The FreeBSD Foundation
2019-05-27 21:36:28 +00:00
Alan Somers
7637cc62ab fusefs: remove obsolete comments in the tests
Sponsored by:	The FreeBSD Foundation
2019-05-27 17:14:46 +00:00
Alan Somers
29edc611c1 fusefs: make the tests more cplusplusy
* Prefer std::unique_ptr to raw pointers
* Prefer pass-by-reference to pass-by-pointer
* Prefer static_cast to C-style cast, unless it's too much typing

Reported by:	ngie
Sponsored by:	The FreeBSD Foundation
2019-05-27 17:08:16 +00:00
Alan Somers
cc04566c46 fusefs: more build fixes
* Fix printf format strings on 32-bit OSes
* Fix -Wclass-memaccess violation on GCC-8 caused by using memset on an object
  of non-trivial type.
* Fix memory leak in MockFS::init
* Fix -Wcast-align error on i386 in expect_readdir
* Fix some heterogenous comparison errors on 32-bit OSes.

Sponsored by:	The FreeBSD Foundation
2019-05-26 03:52:35 +00:00
Alan Somers
93fecd02a1 fusefs: misc build fixes
* Only build the tests on platforms with C++14 support
* Fix an undefined symbol error on lint builds
* Remove an unused function: fiov_clear

Sponsored by:	The FreeBSD Foundation
2019-05-25 21:40:27 +00:00
Alan Somers
e97ae4ad2d fusefs: implement FUSE_ASYNC_READ
If a daemon sets the FUSE_ASYNC_READ flag during initialization, then the
client is allowed to issue multiple concurrent reads for the same file
handle.  Otherwise concurrent reads are not allowed.  This commit implements
it.  Previously we unconditionally disallowed concurrent reads.

Sponsored by:	The FreeBSD Foundation
2019-05-24 05:12:43 +00:00
Alan Somers
e5b50fe736 fusefs: Make fuse file systems NFS-exportable
This commit adds the VOPs needed by userspace NFS servers (tested with
net/unfs3).  More work is needed to make the in-kernel nfsd work, because of
its stateless nature.  It doesn't open files prior to doing I/O.  Also, the
NFS-related VOPs currently ignore the entry cache.

Sponsored by:	The FreeBSD Foundation
2019-05-23 00:44:01 +00:00
Alan Somers
18a2264e27 fusefs: fix "recursing on non recursive lockmgr" panic
When mounted with -o default_permissions and when
vfs.fusefs.data_cache_mode=2, fuse_io_strategy would try to clear the suid
bit after a successful write by a non-owner.  When combined with a
not-yet-committed attribute-caching patch I'm working on, and if the
FUSE_SETATTR response indicates an unexpected filesize (legal, if the file
system has other clients), this would end up calling vtruncbuf.  That would
panic, because the buffer lock was already held by bufwrite or bufstrategy
or something else upstack from fuse_vnop_strategy.

Sponsored by:	The FreeBSD Foundation
2019-05-22 23:30:51 +00:00
Alan Somers
a6fac00c53 fusefs: Allow update mounts
Allow "mount -u" to change some mount options for fusefs.

Sponsored by:	The FreeBSD Foundation
2019-05-21 19:34:39 +00:00
Alan Somers
fe221e0177 fusefs: forward UTIME_NOW to the server
If a user sets both atime and mtime to UTIME_NOW when calling a syscall like
utimensat(2), allow the server to choose what "now" means.  Due to the
design of FreeBSD's VFS, it's not possible to do this for just one of atime
or mtime; it's all or none.

PR:		237181
Sponsored by:	The FreeBSD Foundation
2019-05-16 23:17:39 +00:00
Alan Somers
e7f73af118 fusefs: allow the server to specify st_blksize
If the server sets fuse_attr.blksize to a nonzero value in the response to
FUSE_GETATTR, then the client should use that as the value for
stat.st_blksize .

Sponsored by:	The FreeBSD Foundation
2019-05-16 22:50:04 +00:00
Alan Somers
16bd2d47c7 fusefs: Upgrade FUSE protocol to version 7.9.
This commit upgrades the FUSE API to protocol 7.9 and adds unit tests for
backwards compatibility with servers built for version 7.8.  It doesn't
implement any of 7.9's new features yet.

Sponsored by:	The FreeBSD Foundation
2019-05-16 17:24:11 +00:00
Alan Somers
77fbe6944f fusefs: fix more intermittency in the dev_fuse_poll tests
When using poll, kevent, or select there was a race window during which it
would be impossible to shut down the daemon.  The problem was that poll,
kevent, and select don't return when the file descriptor gets closed (or
maybe it was that the file descriptor got closed before those syscalls were
entered?).  The solution is to impose a timeout on those syscalls, and check
m_quit after they time out.

Sponsored by:	The FreeBSD Foundation
2019-05-15 20:01:41 +00:00
Alan Somers
6fd2d8e148 fusefs: fix some intermittency in the Kqueue.data test
Expect the FUSE_GETATTR operations for bar and baz to come in either order.

Sponsored by:	The FreeBSD Foundation
2019-05-15 19:23:29 +00:00
Alan Somers
c4fbda2b2c fusefs: commit missing file from r347547
Sponsored by:	The FreeBSD Foundation
2019-05-13 19:48:57 +00:00
Alan Somers
0a7c63e075 fusefs: Report the number of available ops in kevent(2)
Just like /dev/devctl, /dev/fuse will now report the number of operations
available for immediate read in the kevent.data field during kevent(2).

Sponsored by:	The FreeBSD Foundation
2019-05-12 15:27:18 +00:00
Alan Somers
3429092cd1 fusefs: support kqueue for /dev/fuse
/dev/fuse was already pollable with poll and select.  Add support for
kqueue, too.  And add tests for polling with poll, select, and kqueue.

Sponsored by:	The FreeBSD Foundation
2019-05-11 22:58:25 +00:00
Alan Somers
a81776c270 fusefs: fix intermittency in the interrupt tests
* In the fatal_signal test, wait for the daemon to receive FUSE_INTERRUPT
  before exiting.
* Explicitly disable restarting syscalls after SIGUSR2.  This fixes
  intermittency in the priority test.  I don't know why, but sometimes that
  test's mkdir would be restarted, and sometimes it would return EINTR.
  ERESTART should be the default.
* Remove a useless copy/pasted sleep in the priority test.

Sponsored by:	The FreeBSD Foundation
2019-05-10 18:18:41 +00:00
Alan Somers
99cf7bff46 fusefs: debugability improvements in the tests
Fix a mislocated statement from r347431, and add more detail for FUSE_MKDIR

Sponsored by:	The FreeBSD Foundation
2019-05-10 18:14:39 +00:00
Alan Somers
81a619c4e1 fusefs: fix intermittency in the Destroy.ok test
The handler for FUSE_DESTROY must shut down the daemon.

Sponsored by:	The FreeBSD Foundation
2019-05-10 16:58:05 +00:00
Alan Somers
7e0aac2408 fusefs: return ENOTCONN instead of EIO if the daemon dies suddenly
If the daemon dies, return ENOTCONN for all operations that have already
been sent to the daemon, as well as any new ones.

Sponsored by:	The FreeBSD Foundation
2019-05-10 16:41:33 +00:00
Alan Somers
fd182076ff fusefs: fix intermittency in the Interrupt.already_complete test
Sponsored by:	The FreeBSD Foundation
2019-05-10 15:55:30 +00:00
Alan Somers
8b73a4c5ae fusefs: fix running multiple daemons concurrently
When a FUSE daemon dies or closes /dev/fuse, all of that daemon's pending
requests must be terminated.  Previously that was done in /dev/fuse's
.d_close method.  However, d_close only gets called on the *last* close of
the device.  That means that if multiple daemons were running concurrently,
all but the last daemon to close would leave their I/O hanging around.  The
problem was easily visible just by running "kyua -v parallelism=2 test" in
fusefs's test directory.

Fix this bug by terminating a daemon's pending I/O during /dev/fuse's
cdvpriv dtor method instead.  That method runs on every close of a file.

Also, fix some potential races in the tests:
* Clear SA_RESTART when registering the daemon's signal handler so read(2)
  will return EINTR.
* Wait for the daemon to die before unmounting the mountpoint, so we won't
  see an unwanted FUSE_DESTROY operation in the mock file system.

Sponsored by:	The FreeBSD Foundation
2019-05-10 15:02:29 +00:00
Alan Somers
a87257ac25 fusefs: shorten and consolidate sleeps
Some fusefs tests must sleep because they deliberately trigger a race, or
because they're testing the cache timeout functionality.  Consolidate the
sleep interval in a single place so it will be easy to adjust.  Shorten it
from either 500ms or 250ms to 100ms.  From experiment I find that 10ms works
every time, so 100ms should be fairly safe.

Sponsored by:	The FreeBSD Foundation
2019-05-09 18:23:09 +00:00
Alan Somers
f528b38f60 fusefs: eliminate some sleeps in the Interrupt tests
Replace some sleeps with semaphore operations.  Not all sleeps can be
replaced, though.  Some are trying to lose a race.

Sponsored by:	The FreeBSD Foundation
2019-05-09 17:57:04 +00:00
Alan Somers
d5ff268834 fusefs: create sockets with FUSE_MKNOD, not FUSE_CREATE
libfuse expects sockets to be created with FUSE_MKNOD, not FUSE_CREATE,
because that's how Linux does it.  My first attempt at creating sockets
(r346894) used FUSE_CREATE because FreeBSD uses VOP_CREATE for this purpose.
There are no backwards-compatibility concerns with this change, because
socket support hasn't yet been merged to head.

Sponsored by:	The FreeBSD Foundation
2019-05-09 16:25:01 +00:00
Alan Somers
002e54b0aa fusefs: clear a dir's attr cache when its contents change
Any change to a directory's contents should cause its mtime and ctime to be
updated by the FUSE daemon.  Clear its attribute cache so we'll get the new
attributs the next time that they're needed.  This affects the following
VOPs: VOP_CREATE, VOP_LINK, VOP_MKDIR, VOP_MKNOD, VOP_REMOVE, VOP_RMDIR, and
VOP_SYMLINK

Reported by:	pjdfstest
Sponsored by:	The FreeBSD Foundation
2019-05-09 01:16:34 +00:00
Alan Somers
8e45ec4e64 fusefs: fix a permission handling bug during VOP_RENAME
If the file to be renamed is a directory and it's going to get a new parent,
then the user must have write permissions to that directory, because the
".." dirent must be changed.

Reported by:	pjdfstest
Sponsored by:	The FreeBSD Foundation
2019-05-08 22:28:13 +00:00