In libc++'s __threading_support header the semaphore.h header was
implicitly included, but from version 14 onwards, this is no longer the
case, resulting in compile errors:
tests/sys/fs/fusefs/setattr.cc:740:8: error: variable has incomplete type 'sem_t' (aka '_sem')
sem_t sem;
^
tests/sys/fs/fusefs/utils.hh:33:8: note: forward declaration of '_sem'
struct _sem;
^
MFC after: 3 days
Now posix_fallocate will be correctly forwarded to fuse file system
servers, for those that support it.
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D33389
By default, FUSE file systems are assumed not to support lookups for "."
and "..". They must opt-in to that. To cope with this limitation, the
fusefs kernel module caches every fuse vnode's parent's inode number,
and uses that during VOP_LOOKUP for "..". But if the parent's vnode has
been reclaimed that won't be possible. Previously we paniced in this
situation. Now, we'll return ESTALE instead. Or, if the file system
has opted into ".." lookups, we'll just do that instead.
This commit also fixes VOP_LOOKUP to respect the cache timeout for ".."
lookups, if the FUSE file system specified a finite timeout.
PR: 259974
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D33239
In an earlier version of the revision that created that sysctl (D20519)
the sysctl was gated by INVARIANTS, so the test had to check for it.
But in the committed version it is always available.
MFC after: 2 weeks
If FUSE_COPY_FILE_RANGE returns successfully, update the atime of the
source and the mtime and ctime of the destination.
MFC after: 2 weeks
Reviewers: pfg
Differential Revision: https://reviews.freebsd.org/D33159
VOPs like VOP_SETATTR can change a file's size, with the vnode
exclusively locked. But VOPs like VOP_LOOKUP look up the file size from
the server without the vnode locked. So a race is possible. For
example:
1) One thread calls VOP_SETATTR to truncate a file. It locks the vnode
and sends FUSE_SETATTR to the server.
2) A second thread calls VOP_LOOKUP and fetches the file's attributes from
the server. Then it blocks trying to acquire the vnode lock.
3) FUSE_SETATTR returns and the first thread releases the vnode lock.
4) The second thread acquires the vnode lock and caches the file's
attributes, which are now out-of-date.
Fix this race by recording a timestamp in the vnode of the last time
that its filesize was modified. Check that timestamp during VOP_LOOKUP
and VFS_VGET. If it's newer than the time at which FUSE_LOOKUP was
issued to the server, ignore the attributes returned by FUSE_LOOKUP.
PR: 259071
Reported by: Agata <chogata@moosefs.pro>
Reviewed by: pfg
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D33158
FUSE_COPY_FILE_RANGE instructs the server to write data to a file.
fusefs must invalidate any cached data within the written range.
PR: 260242
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D33280
Correctly handle the situation where a FUSE server unlinks a file, then
creates a new file of a different type but with the same inode number.
Previously fuse_vnop_lookup in this situation would return EAGAIN. But
since it didn't call vgone(), the vnode couldn't be reused right away.
Fix this by immediately calling vgone() and reallocating a new vnode.
This problem can occur in three code paths, during VOP_LOOKUP,
VOP_SETATTR, or following FUSE_GETATTR, which usually happens during
VOP_GETATTR but can occur during other vops, too. Note that the correct
response actually doesn't depend on whether the entry cache has expired.
In fact, during VOP_LOOKUP, we can't even tell. Either it has expired
already, or else the vnode got reclaimed by vnlru.
Also, correct the error code during the VOP_SETATTR path.
PR: 258022
Reported by: chogata@moosefs.pro
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D33283
When using cached attributes, whether or not the data cache is enabled,
fusefs must update a file's atime whenever it reads from it, so long as
it wasn't mounted with -o noatime. Update it in-kernel, and flush it to
the server on close or during the next setattr operation.
The downside is that close() will now frequently trigger a FUSE_SETATTR
upcall. But if you care about performance, you should be using
-o noatime anyway.
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D33145
When copy_file_range extends a file, it must update the cached file
size.
MFC after: 2 weeks
Reviewed by: rmacklem, pfg
Differential Revision: https://reviews.freebsd.org/D33151
The DevFusePoll::access/select test would occasionally segfault. The
cause was a file descriptor that was shared between two threads. The
first thread would kill the second and close the file descriptor. But
it was possible that the second would read the file descriptor before it
shut down. That did not cause problems for kqueue, poll, or blocking
operation, but it triggered segfaults in select's macros.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D32142
If the FUSE server tells the kernel that a file's size has changed, then
the kernel must invalidate any portion of that file in cache. But the
kernel can't do that during VOP_STRATEGY, because the file's buffers are
already locked. Instead, proceed with the write.
PR: 256937
Reported by: Agata <chogata@moosefs.pro>
Tested by: Agata <chogata@moosefs.pro>
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D32332
fuse_vnop_bmap needs to know the file's size in order to calculate the
optimum amount of readahead. If the file's size is unknown, it must ask
the FUSE server. But if the file's data was previously cached and the
server reports that its size has shrunk, fusefs must invalidate the
cached data. That's not possible during VOP_BMAP because the buffer
object is already locked.
Fix the panic by not querying the FUSE server for the file's size during
VOP_BMAP if we don't need it. That's also a a slight performance
optimization.
PR: 256937
Reported by: Agata <chogata@moosefs.pro>
Tested by: Agata <chogata@moosefs.pro>
MFC after: 2 weeks
For file systems that allow it, fusefs will skip FUSE_OPEN,
FUSE_RELEASE, FUSE_OPENDIR, and FUSE_RELEASEDIR operations, a minor
optimization.
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D32141
There sig_atomic_t is shorter than void *.
As result, it cannot keep pointer.
Assigning to void * is actually safe for us in a signal handler.
Reviewed by: asomers
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Fixes: 4f917847c9
Differential revision: https://reviews.freebsd.org/D32064
During VOP_GETPAGES, fusefs needs to determine the file's length, which
could require a FUSE_GETATTR operation. If that fails, it's better to
SIGBUS than panic.
MFC after: 1 week
Sponsored by: Axcient
Reviewed by: markj, kib
Differential Revision: https://reviews.freebsd.org/D31994
Follow-up d396c67f26 by also silencing warnings about deprecated
implicit copy constructors in the fusefs tests, which use googletest.
Fixes: d396c67f26
MFC after: 3 days
During FUSE_SETLK, the owner field should uniquely identify the calling
process. The fusefs module now sets it to the process's pid.
Previously, it expected the calling process to set it directly, which
was wrong.
libfuse also apparently expects the owner field to be set during
FUSE_GETLK, though I'm not sure why.
PR: 256005
Reported by: Agata <chogata@moosefs.pro>
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D30622
Every FUSE operation has a unique value in its header. As the name
implies, these values are supposed to be unique among all outstanding
operations. And since FUSE_INTERRUPT is asynchronous and racy, it is
desirable that the unique values be unique among all operations that are
"close in time".
Ensure that they are actually unique by incrementing them whenever we
reuse a fuse_dispatcher object, for example during fsync, write, and
listextattr.
PR: 244686
MFC after: 2 weeks
Reviewed by: pfg
Differential Revision: https://reviews.freebsd.org/D30810
/dev/fuse is always ready for writing, so it's kind of dumb to poll it.
But some applications do it anyway. Better to return ready than EINVAL.
MFC after: 2 weeks
Reviewed by: emaste, pfg
Differential Revision: https://reviews.freebsd.org/D30784
1) F_SETLKW (blocking) operations would be sent to the FUSE server as
F_SETLK (non-blocking).
2) Release operations, F_SETLK with lk_type = F_UNLCK, would simply
return EINVAL.
PR: 253500
Reported by: John Millikin <jmillikin@gmail.com>
MFC after: 2 weeks
This allows d_off to be used with lseek to position the file so that
getdirentries(2) will return the next entry. It is not used by
readdir(3).
PR: 253411
Reported by: John Millikin <jmillikin@gmail.com>
Reviewed by: cem
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28605
This updates the FUSE protocol to 7.28, though most of the new features
are optional and are not yet implemented.
MFC after: 2 weeks
Relnotes: yes
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27818
An order-of-operations problem caused an expectation intended for
FUSE_READ to instead match FUSE_ACCESS. Surprisingly, only one test
case was affected.
MFC after: 2 weeks
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27818
FUSE_LSEEK reports holes on fuse file systems, and is used for example
by bsdtar.
MFC after: 2 weeks
Relnotes: yes
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27804
maxphys is now a tunable, ever since r368124. The default value is also
larger than it used to be. That broke several fusefs tests that made
assumptions about maxphys.
* WriteCluster.clustering used the MAXPHYS compile-time constant.
* WriteBackAsync.direct_io_partially_overlaps_cached_block implicitly
depended on the default value of maxphys. Fix it by making the
dependency explicit.
* Write.write_large implicitly assumed that maxphys would be no more
than twice maxbcachebuf. Fix it by explicitly setting m_max_write.
* WriteCluster.clustering and several others failed because the MockFS
module did not work for max_write > 128KB (which most tests would set
when maxphys > 256KB). Limit max_write accordingly. This is the same
as fusefs-libs's behavior.
* Bmap's tests were originally written for MAXPHYS=128KB. With larger
values, the simulated file size was too small.
PR: 252096
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D27769
Repeating the default WARNS here makes it slightly more difficult to
experiment with default WARNS changes, e.g. if we did something absolutely
bananas and introduced a WARNS=7 and wanted to try lifting the default to
that.
Drop most of them; there is one in the blake2 kernel module, but I suspect
it should be dropped -- the default WARNS in the rest of the build doesn't
currently apply to kernel modules, and I haven't put too much thought into
whether it makes sense to make it so.
If a FUSE server returns FOPEN_DIRECT_IO in response to FUSE_OPEN, that
instructs the kernel to bypass the page cache for that file. This feature
is also known by libfuse's name: "direct_io".
However, when accessing a file via mmap, there is no possible way to bypass
the cache completely. This change fixes a deadlock that would happen when
an mmap'd write tried to invalidate a portion of the cache, wrongly assuming
that a write couldn't possibly come from cache if direct_io were set.
Arguably, we could instead disable mmap for files with FOPEN_DIRECT_IO set.
But allowing it is less likely to cause user complaints, and is more in
keeping with the spirit of open(2), where O_DIRECT instructs the kernel to
"reduce", not "eliminate" cache effects.
PR: 247276
Reported by: trapexit@spawn.link
Reviewed by: cem
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D26485
Thanks to r364064, the name cache now returns a hit where previously it
would miss. Adjust the expectations accordingly.
PR: 248583
Reported by: lwhsu
MFC with: r364064
This patch fixes two issues relating to FUSE_ACCESS when the
default_permissions mount option is disabled:
* VOP_ACCESS() calls with VADMIN set should never be sent to a fuse server
in the form of FUSE_ACCESS operations. The FUSE protocol has no equivalent
of VADMIN, so we must evaluate such things kernel-side, regardless of the
default_permissions setting.
* The FUSE protocol only requires FUSE_ACCESS to be sent for two purposes:
for the access(2) syscall and to check directory permissions for
searchability during lookup. FreeBSD sends it much more frequently, due to
differences between our VFS and Linux's, for which FUSE was designed. But
this patch does eliminate several cases not required by the FUSE protocol:
* for any FUSE_*XATTR operation
* when creating a new file
* when deleting a file
* when setting timestamps, such as by utimensat(2).
* Additionally, when default_permissions is disabled, this patch removes one
FUSE_GETATTR operation when deleting a file.
PR: 245689
Reported by: MooseFS FreeBSD Team <freebsd@moosefs.pro>
Reviewed by: cem
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D24777
When a FUSE operation other than LOOKUP returns ENOENT, the kernel will
reclaim that vnode, resuling in a FUSE_FORGET being sent a short while
later. Many of the ENOENT tests weren't expecting those FUSE_FORGET
operations. They usually passed by luck since FUSE_FORGET is often delayed.
This commit adds appropriate expectations.
MFC after: 2 weeks
These two errors have been present since the tests' introduction.
Coincidentally every test (I think there's only one) that cares about that
field also works when the field's value is 0.
MFC after: 2 weeks
mac_bsdextended(4), when enabled, causes ordinary operations to send many
more VOP_GETATTRs to file system. The fusefs tests expectations aren't
written with those in mind. Optionally expecting them would greatly
obfuscate the fusefs tests. Worse, certain fusefs functionality (like
attribute caching) would be impossible to test if the tests couldn't expect
an exact number of GETATTR operations.
This commit resolves that conflict by making two changes:
1. The fusefs tests will now check for mac_bsdextended, and skip if it's
enabled.
2. The mac_bsdextended tests will now check whether the module is enabled, not
merely loaded. If it's loaded but disabled, the tests will automatically
enable it for the duration of the tests.
With these changes, a CI system can achieve best coverage by loading both
fusefs and mac_bsdextended at boot, and setting
security.mac.bsdextended.enabled=0
PR: 244229
Reported by: lwhsu
Reviewed by: cem
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D24577
The FUSE protocol allows the client (kernel) to cache a file's size, if the
server (userspace daemon) allows it. A well-behaved daemon obviously should
not change a file's size while a client has it cached. But a buggy daemon
might. If the kernel ever detects that that has happened, then it should
invalidate the entire cache for that file. Previously, we would not only
cache stale data, but in the case of a file extension while we had the size
cached, we accidentally extended the cache with zeros.
PR: 244178
Reported by: Ben RUBSON <ben.rubson@gmx.com>
Reviewed by: cem
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D24012
We were reusing a structure for multiple operations, but failing to
reinitialize one member. The result is that a server that cares about FUSE
file handle IDs would see one correct FUSE_FSYNC operation, and one with the
FHID unset.
PR: 244431
Reported by: Agata <chogata@gmail.com>
MFC after: 2 weeks
This re-enables building the googletest suite by default on mips and instead
specifically doesn't build fusefs tests for mips+clang builds. clang will
easily spent >= 1.5 hours compiling a single file due to a bug in
optimization (see LLVM PR 43263), so turn these off for now while that's
hashed out.
GCC builds are unaffected and build the fusefs tests as-is. Clang builds
only happen by early adopters attempting to hash out the remaining issues.
The comment has been updated to reflect its new position and use less strong
wording about imposing on people.
Discussed with: ngie, asomers
Reviewed by: ngie
Coverity complained that I wasn't initializing some class members until the
SetUp method. Do it in the constructor instead.
Reported by: Coverity
Coverity CIDs: 1404352, 1404378
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
