If the fuse daemon supports FUSE_BMAP, then use that for the block mapping.
Otherwise, use the same technique used by vop_stdbmap. Report large values
for runp and runb in order to maximize read clustering and minimize upcalls,
even if we don't know the true layout.
The major result of this change is that sequential reads to FUSE files will
now usually happen 128KB at a time instead of 64KB.
Sponsored by: The FreeBSD Foundation
* Don't always write the last page synchronously. That's not actually
required. It was probably just masking another bug that I fixed later,
possibly in r349021.
* Enable the NotifyWriteback tests now that Writeback cache is working.
* Add a test to ensure that the write cache isn't flushed synchronously when
in writeback mode.
Sponsored by: The FreeBSD Foundation
fusefs will now use cluster_read. This allows readahead of more than one
cache block. However, it won't yet actually cluster the reads because that
requires VOP_BMAP, which fusefs does not yet implement.
Sponsored by: The FreeBSD Foundation
rename the source to gsb_crc32.c.
This is a prerequisite of unifying kernel zlib instances.
PR: 229763
Submitted by: Yoshihiro Ota <ota at j.email.ne.jp>
Differential Revision: https://reviews.freebsd.org/D20193
fusefs will now read ahead at most one cache block at a time (usually 64
KB). Clustered reads are still TODO. Individual file systems may disable
read ahead by setting fuse_init_out.max_readahead=0 during initialization.
Sponsored by: The FreeBSD Foundation
Our fusefs(5) module supports three cache modes: uncached, write-through,
and write-back. However, the write-through mode (which is the default) has
never actually worked as its name suggests. Rather, it's always been more
like "write-around". It wrote directly, bypassing the cache. The cache
would only be populated by a subsequent read of the same data.
This commit fixes that problem. Now the write-through mode works as one
would expect: write(2) immediately adds data to the cache and then blocks
while the daemon processes the write operation.
A side effect of this change is that non-cache-block-aligned writes will now
incur a read-modify-write cycle of the cache block. The old behavior
(bypassing write cache entirely) can still be achieved by opening a file
with O_DIRECT.
PR: 237588
Sponsored by: The FreeBSD Foundation
Enable write clustering in fusefs whenever cache mode is set to writeback
and the "async" mount option is used. With default values for MAXPHYS,
DFLTPHYS, and the fuse max_write mount parameter, that means sequential
writes will now be written 128KB at a time instead of 64KB.
Also, add a regression test for PR 238565, a panic during unmount that
probably affects UFS, ext2, and msdosfs as well as fusefs.
PR: 238565
Sponsored by: The FreeBSD Foundation
An errant vfs_bio_clrbuf snuck in in r348931. Surprisingly, it doesn't have
any effect most of the time. But under some circumstances it cause the
buffer to behave in a write-only fashion.
Sponsored by: The FreeBSD Foundation
The current "writeback" cache mode, selected by the
vfs.fusefs.data_cache_mode sysctl, doesn't do writeback cacheing at all. It
merely goes through the motions of using buf(9), but then writes every
buffer synchronously. This commit:
* Enables delayed writes when the sysctl is set to writeback cacheing
* Fixes a cache-coherency problem when extending a file whose last page has
just been written.
* Removes the "sync" mount option, which had been set unconditionally.
* Adjusts some SDT probes
* Adds several new tests that mimic what fsx does but with more control and
without a real file system. As I discover failures with fsx, I add
regression tests to this file.
* Adds a test that ensures we can append to a file without reading any data
from it.
This change is still incomplete. Clustered writing is not yet supported,
and there are frequent "panic: vm_fault_hold: fault on nofault entry" panics
that I need to fix.
Sponsored by: The FreeBSD Foundation
fusefs's I/O methods were originally copy/pasted from nfsclient. This
commit removes some irrelevant parts, like stuff involving B_NEEDCOMMIT.
Sponsored by: The FreeBSD Foundation
Both filesystems do no use vnode_pager_dealloc() which would handle
this case otherwise. Nullfs because vnode vm_object handle never
points to nullfs vnode. Tmpfs because its vm_object is never vnode
object at all.
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
In r348560 I thought that FUSE_EXPORT_SUPPORT was required for cases where
the node to be invalidated (or the parent of the entry to be invalidated)
wasn't cached. But I realize now that that's not the case. During entry
invalidation, if the parent isn't in the vfs hash table, then it must've
been reclaimed. And since fuse_vnop_reclaim does a cache_purge, that means
the entry to be invalidated has already been removed from the namecache.
And during inode invalidation, if the inode to be invalidated isn't in the
vfs hash table, then it too must've been reclaimed. In that case it will
have no buffer cache to invalidate.
Sponsored by: The FreeBSD Foundation
Protocol 7.12 adds a way for the server to notify the client that it should
invalidate an inode's data cache and/or attributes. This commit implements
that mechanism. Unlike Linux's implementation, ours requires that the file
system also supports FUSE_EXPORT_SUPPORT (NFS-style lookups). Otherwise the
invalidation operation will return EINVAL.
Sponsored by: The FreeBSD Foundation
Protocol 7.12 adds a way for the server to notify the client that it should
invalidate an entry from its name cache. This commit implements that
mechanism.
Sponsored by: The FreeBSD Foundation
FUSE allows entries to be cached for a limited amount of time. fusefs's
vnop_lookup method already implements that using the timeout functionality
of cache_lookup/cache_enter_time. However, lookups for the NFS server go
through a separate path: vfs_vget. That path can't use the same timeout
functionality because cache_lookup/cache_enter_time only work on pathnames,
whereas vfs_vget works by inode number.
This commit adds entry timeout information to the fuse vnode structure, and
checks it during vfs_vget. This allows the NFS server to take advantage of
cached entries. It's also the same path that FUSE's asynchronous cache
invalidation operations will use.
Sponsored by: The FreeBSD Foundation
This silly code segment has existed in the sources since it was brought
into FreeBSD 10 years ago. I honestly have no idea why this was done.
It was possible that I thought that it might have been better to not
set B_ASYNC for the "else" case, but I can't remember.
Anyhow, this patch gets rid of the if/else that does the same thing
either way, since it looks silly and upsets a static analyser.
This will have no semantic effect on the NFS client.
PR: 238167
This commit raises the protocol level and adds backwards-compatibility code
to handle structure size changes. It doesn't implement any new features.
The new features added in protocol 7.12 are:
* server-side umask processing (which FreeBSD won't do)
* asynchronous inode and directory entry invalidation (which I'll do next)
Sponsored by: The FreeBSD Foundation
Protocol 7.11 adds two new features, but neither of them were defined
correctly. FUSE_IOCTL messages don't work for 32-bit daemons on a 64-bit
host (fixed in protocol 7.16). FUSE_POLL is basically unusable until 7.21.
Before 7.21, the client can't choose which events to register for; the
client registers for "something" and the server replies to say which events
the client is registered for. Also, before 7.21 there was no way for a
client to deregister a file handle.
Sponsored by: The FreeBSD Foundation
This commit adds the definitions for protocol 7.11 but doesn't yet implement
the new features. The new features are optional, so they can come later.
Sponsored by: The FreeBSD Foundation
Protocol version 7.10 has only one new feature, and I'm choosing not to
implement it, so this commit is basically a noop. The sole new feature is
the FOPEN_NONSEEKABLE flag, which a fuse file system can return to indicate
that a certain file handle cannot be seeked. However, I'm unaware of any
file system in ports that uses this flag.
Sponsored by: The FreeBSD Foundation
Users of pseudofs (e.g. lindebugfs), should be able to receive
input from command line via commands like "echo 1 > /path/to/file".
Currently this fails because sh tries to truncate the file first and
vop_setattr returns not supported error for this. This patch simply
ignores the error and returns 0 instead.
Reviewed by: imp (mentor), asomers
Approved by: imp (mentor), asomers
MFC after: 1 week
Differential Revision: D20451
These fields are supposed to contain the file descriptor flags as supplied
to open(2) or set by fcntl(2). The feature is kindof useless on FreeBSD
since we don't supply all of these flags to fuse (because of the weak
relationship between struct file and struct vnode). But we should at least
set the access mode flags (O_RDONLY, etc).
This is the last fusefs change needed to get full protocol 7.9 support.
There are still a few options we don't support for good reason (mandatory
file locking is dumb, flock support is broken in the protocol until 7.17,
etc), but there's nothing else to do at this protocol level.
Sponsored by: The FreeBSD Foundation
If a FUSE file system sets the FUSE_POSIX_LOCKS flag then it can support
fcntl(2)-style locks directly. However, the protocol does not adequately
support flock(2)-style locks until revision 7.17. They must be implemented
locally in-kernel instead. This unfortunately breaks the interoperability
of fcntl(2) and flock(2) locks for file systems that support the former.
C'est la vie.
Prior to this commit flock(2) would get sent to the server as a
fcntl(2)-style lock with the lock owner field set to stack garbage.
Sponsored by: The FreeBSD Foundation
Protocol 7.9 adds this field. We could use it to store the file handle of
the file whose attributes we're requesting. However, that requires extra
work at runtime to look up a file handle, and I'm not aware of any file
systems that care. So it's easiest just to clear it.
Sponsored by: The FreeBSD Foundation
This bit tells the server that we're not sure which uid, gid, and/or pid
originated the write. I don't know of a single file system that cares, but
it's part of the protocol.
Sponsored by: The FreeBSD Foundation
* Only build the tests on platforms with C++14 support
* Fix an undefined symbol error on lint builds
* Remove an unused function: fiov_clear
Sponsored by: The FreeBSD Foundation
vtruncbuf takes a "struct ucred*" argument. AFAICT, it's been unused ever
since that function was first added in r34611. Remove it. Also, remove some
"struct ucred" arguments from fuse and nfs functions that were only used by
vtruncbuf.
Reviewed by: cem
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20377
If a daemon sets the FUSE_ASYNC_READ flag during initialization, then the
client is allowed to issue multiple concurrent reads for the same file
handle. Otherwise concurrent reads are not allowed. This commit implements
it. Previously we unconditionally disallowed concurrent reads.
Sponsored by: The FreeBSD Foundation
A previous commit made fuse exportable via userland NFS servers.
Compatibility with the in-kernel nfsd required two more changes:
* During read and write operations, implicitly do a FUSE_OPEN if there isn't
already a valid file handle. That's because nfsd never calls VOP_OPEN.
* During VOP_READDIR, if an implicit open was necessary, directory offsets
from a previous VOP_READDIR may not be valid, so VOP_READDIR may have to
start from the beginning and read until it encounters the requested
offset.
I've done only limited testing over NFS, so there are probably still some
more bugs. Thanks to rmacklem for all of the readdir changes, which he had
made for his pnfs work.
Sponsored by: The FreeBSD Foundation
This commit adds the VOPs needed by userspace NFS servers (tested with
net/unfs3). More work is needed to make the in-kernel nfsd work, because of
its stateless nature. It doesn't open files prior to doing I/O. Also, the
NFS-related VOPs currently ignore the entry cache.
Sponsored by: The FreeBSD Foundation
Consolidate all calls to fuse_vnode_setsize as a result of a file attribute
change to one location in fuse_internal_setattr. There are still a few
calls elsewhere that happen as a result of a write.
Sponsored by: The FreeBSD Foundation
When mounted with -o default_permissions and when
vfs.fusefs.data_cache_mode=2, fuse_io_strategy would try to clear the suid
bit after a successful write by a non-owner. When combined with a
not-yet-committed attribute-caching patch I'm working on, and if the
FUSE_SETATTR response indicates an unexpected filesize (legal, if the file
system has other clients), this would end up calling vtruncbuf. That would
panic, because the buffer lock was already held by bufwrite or bufstrategy
or something else upstack from fuse_vnop_strategy.
Sponsored by: The FreeBSD Foundation
In r347547 I intended to remove the vfs.fusefs.sync_resize sysctl, leaving
fusefs's behavior as though sync_resize had its default value. But I forgot
that I had already turned off sync_resize in my development system's
/etc/sysctl.conf.
This commit complete removes the optional behavior that was formerly
controlled by sync_resize. There's no need for explicitly calling
FUSE_SETATTR after every FUSE_WRITE that extends a file. The daemon can
infer that the file is being extended. If this sysctl was added as a
workaround for a buggy daemon, there's no clue as to what that daemon may
have been.
Sponsored by: The FreeBSD Foundation
Similar to r348026, exhaustive search for uses of CTRn() and cross reference
ktr.h includes. Where it was obvious that an OS compat header of some kind
included ktr.h indirectly, .c files were left alone. Some of these files
clearly got ktr.h via header pollution in some scenarios, or tinderbox would
not be passing prior to this revision, but go ahead and explicitly include it
in files using it anyway.
Like r348026, these CUs did not show up in tinderbox as missing the include.
Reported by: peterj (arm64/mp_machdep.c)
X-MFC-With: r347984
Sponsored by: Dell EMC Isilon
The kernel can't tell whether or not a fuse file system is truly local. But
what really matters is two things:
1) Can I/O to a file system block indefinitely?
2) Can the file system bypass the O_BENEATH restriction during lookup?
For fuse, the answer to both of those question is yes. So as far as the
kernel is concerned, it's a non-local file system.
Sponsored by: The FreeBSD Foundation
This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.
EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).
As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions. The remainder of the patch addresses
adding appropriate includes to fix those files.
LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).
No functional change (intended). Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed. __FreeBSD_version has been bumped.
If a user sets both atime and mtime to UTIME_NOW when calling a syscall like
utimensat(2), allow the server to choose what "now" means. Due to the
design of FreeBSD's VFS, it's not possible to do this for just one of atime
or mtime; it's all or none.
PR: 237181
Sponsored by: The FreeBSD Foundation
If the server sets fuse_attr.blksize to a nonzero value in the response to
FUSE_GETATTR, then the client should use that as the value for
stat.st_blksize .
Sponsored by: The FreeBSD Foundation
This commit upgrades the FUSE API to protocol 7.9 and adds unit tests for
backwards compatibility with servers built for version 7.8. It doesn't
implement any of 7.9's new features yet.
Sponsored by: The FreeBSD Foundation
fuse_kernel.h defines the structures used by the FUSE protocol. Originally
it came from libfuse, but the current source of truth is the Linux kernel.
This commit minimizes the diffs between our version and the Linux version as
of 21f3da95d (protocol version 7.8).
The flags field of struct fuse_listxattr_out and fuse_listxattr_in was an
error in our header. Those fields don't exist in Linux or libfuse, and
they've never been used in FreeBSD. In fact, those structs don't even exist
in Linux and libfuse; those projects confusingly overload the identical
fuse_getexattr_in and fuse_getxattr_out structs.
Sponsored by: The FreeBSD Foundation
